Sep 08 15:30:33.643678 ip-10-0-135-240 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 08 15:30:33.643688 ip-10-0-135-240 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 08 15:30:33.643696 ip-10-0-135-240 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 08 15:30:33.644031 ip-10-0-135-240 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 08 15:30:43.723880 ip-10-0-135-240 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 08 15:30:43.723899 ip-10-0-135-240 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6ea2ac04f77c4c9f945fcacaf4cee516 --
Sep 08 15:32:33.447657 ip-10-0-135-240 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 15:32:33.886866 ip-10-0-135-240 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:32:33.886866 ip-10-0-135-240 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 15:32:33.886866 ip-10-0-135-240 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:32:33.886866 ip-10-0-135-240 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 15:32:33.886866 ip-10-0-135-240 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 15:32:33.889255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.889158    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 15:32:33.897888 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897860    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:32:33.897888 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897885    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:32:33.897888 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897889    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:32:33.897888 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897893    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897896    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897900    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897903    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897906    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897908    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897911    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897914    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897917    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897919    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897922    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897925    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897927    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897930    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897934    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897939    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897943    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897948    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897951    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:32:33.898019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897954    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897958    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897963    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897966    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897970    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897973    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897976    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897979    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897981    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897985    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897988    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897991    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897994    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.897997    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898000    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898003    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898006    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898011    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898014    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:32:33.898490 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898017    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898020    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898023    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898026    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898030    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898033    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898035    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898038    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898040    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898043    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898046    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898048    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898051    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898054    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898056    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898060    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898063    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898065    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898068    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898071    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:32:33.899076 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898074    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898076    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898079    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898082    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898085    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898087    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898090    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898093    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898096    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898099    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898103    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898105    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898108    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898111    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898113    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898116    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898118    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898121    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898123    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898126    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:32:33.899628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898128    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898131    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898133    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898137    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898140    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898142    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898145    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898148    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898151    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898153    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898156    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898158    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898161    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898164    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898167    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898169    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898172    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898174    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898943    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898951    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:32:33.900132 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898955    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898958    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898961    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898971    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898974    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898977    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898979    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898982    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898985    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898988    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898990    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898993    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898996    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.898999    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899002    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899005    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899007    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899010    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899012    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:32:33.900657 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899024    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899027    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899029    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899032    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899034    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899037    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899040    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899042    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899045    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899047    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899050    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899053    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899056    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899060    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899062    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899065    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899068    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899076    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899079    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899082    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:32:33.901142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899085    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899088    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899090    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899093    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899095    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899098    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899101    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899104    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899106    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899109    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899111    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899114    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899117    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899119    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899122    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899125    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899127    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899130    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899132    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899135    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:32:33.901664 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899137    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899140    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899151    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899155    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899159    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899162    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899165    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899168    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899172    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899176    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899180    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899183    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899185    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899188    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899190    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899193    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899195    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899198    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899200    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:32:33.902153 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899203    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899205    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899208    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899210    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899213    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899215    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899219    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899222    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899225    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899228    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899232    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899235    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899239    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899242    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899244    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899247    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899250    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899254    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899256    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899700    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:32:33.902638 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899705    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899708    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899711    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899714    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899717    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899720    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899722    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899726    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899730    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899733    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899735    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899738    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899741    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899744    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899746    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899749    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899751    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899754    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899756    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899759    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:32:33.903142 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899762    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899765    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899767    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899770    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899773    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899776    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899779    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899781    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899784    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899787    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899789    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899792    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899795    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899798    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899802    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899806    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899809    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899813    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899815    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899818    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:32:33.903653 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899821    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899823    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899826    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899829    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899832    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899835    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899837    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899840    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899842    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899845    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899848    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899850    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899853    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899856    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899858    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899861    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899864    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899866    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899869    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899872    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:32:33.904152 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899874    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899877    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899879    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899882    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899884    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899887    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899889    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899893    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899896    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899899    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899902    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899906    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899908    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899911    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899913    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899916    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899918    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899921    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899923    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:32:33.904699 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899926    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899928    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899931    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899934    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899936    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899939    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899942    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899945    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899947    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899950    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899953    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899955    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899958    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899961    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899963    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899965    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899968    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899971    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.899974    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900058    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 15:32:33.905177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900094    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900101    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900106    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900111    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900114    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900127    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900138    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900142    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900145    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900148    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900152    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900155    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900158    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900161    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900164    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900167    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900170    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900177    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900180    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900183    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900186    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900190    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900194    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 15:32:33.905694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900197    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900200    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900203    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900206    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900209    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900212    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900216    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900219    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900223    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900226    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900229    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900232    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900235    2563 flags.go:64] FLAG: --enable-server="true"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900239    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900244    2563 flags.go:64] FLAG: --event-burst="100"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900247    2563 flags.go:64] FLAG: --event-qps="50"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900250    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900253    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900256    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900262    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900265    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900268    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900271    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900274    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900278    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 15:32:33.906268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900281    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900300    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900303    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900306    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900310    2563 flags.go:64] FLAG: --feature-gates=""
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900314    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900317    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900321    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900324    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900327    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900330    2563 flags.go:64] FLAG: --help="false"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900333    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-135-240.ec2.internal"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900336    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900339    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900342    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900346    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900349    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900352    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900355    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900358    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900361    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900364    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900367    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900370    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 08 15:32:33.906931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900374    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900376    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900381    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900384    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900387    2563 flags.go:64] FLAG: --lock-file=""
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900390    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900393    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900396    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900402    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900406    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900409    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900412    2563 flags.go:64] FLAG: --logging-format="text"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900415    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900419    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900422    2563 flags.go:64] FLAG: --manifest-url=""
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900425    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900430    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900433    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900438    2563 flags.go:64] FLAG: --max-pods="110"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900441    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900444    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900448    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900451    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900454    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900457    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 15:32:33.907699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900460    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900468    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900471    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900474    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900477    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900480    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900486    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900489    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900493    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900498    2563 flags.go:64] FLAG: --port="10250"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900501    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900504    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aa9c7148a904e11a"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900507    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900510    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900513    2563 flags.go:64] FLAG: --register-node="true"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900516    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900520    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900525    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900527    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900530    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900534    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900537    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900540    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900543    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900546    2563 flags.go:64] FLAG: --runonce="false"
Sep 08 15:32:33.908395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900549    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900552    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900556    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900558    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900561    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900565    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900568    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900572    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900575    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900578    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900581    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900583    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900587    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900590    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900593    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900599    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900603    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900606    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900610    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900613    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900615    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900618    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900621    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900624    2563 flags.go:64] FLAG: --v="2"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900630    2563 flags.go:64] FLAG: --version="false"
Sep 08 15:32:33.909018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900634    2563 flags.go:64] FLAG: --vmodule=""
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900638    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.900641    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900778    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900783    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900786    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900788    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900791    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900794    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900797    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900799    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900802    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900805    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900808    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900810    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900813    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900816    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900818    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900821    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:32:33.909693 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900825    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900829    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900834    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900837    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900842    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900845    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900847    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900850    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900853    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900856    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900858    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900861    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900865    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900869    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900872    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900874    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900877    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900880    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900883    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900886    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:32:33.910181 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900889    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900892    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900894    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900897    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900909    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900912    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900915    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900917    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900920    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900923    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900925    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900928    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900931    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900933    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900936    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900940    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900945    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900947    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900950    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900953    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:32:33.910701 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900956    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900958    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900961    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900964    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900968    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900971    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900974    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900976    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900979    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900982    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900985    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900987    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900990    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900993    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900996    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.900998    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901001    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901005    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901009    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:32:33.911197 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901011    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901014    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901017    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901019    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901022    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901025    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901027    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901030    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901033    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901038    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901040    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901043    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901045    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901048    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901051    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901053    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901056    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901061    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901064    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901067    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:32:33.911688 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901069    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901072    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901075    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.901077    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.901083    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.909416    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.909436    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.909458    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.909463    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909530    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909536    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909540    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909543    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909546    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909549    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:32:33.912233 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909552    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909554    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909557    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909560    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909562    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909565    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909576    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909580    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909583    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909586    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909588    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909591    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909593    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909596    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909598    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909601    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909604    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909608    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909613    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:32:33.912646 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909616    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909619    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909621    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909625    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909629    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909631    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909634    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909636    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909640    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909642    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909645    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909647    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909650    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909653    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909656    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909658    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909675    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909679    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909682    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:32:33.913125 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909685    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909695    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909698    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909701    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909704    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909707    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909712    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909715    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909718    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909721    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909724    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909727    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909730    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909733    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909736    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909739    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909741    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909744    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909747    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:32:33.913628 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909750    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909752    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909755    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909758    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909760    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909763    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909765    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909768    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909770    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909773    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909776    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909778    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909781    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909784    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909786    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909794    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909797    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909800    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909803    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909806    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:32:33.914090 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909808    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909811    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909813    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909816    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909818    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909821    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909824    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909826    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909829    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909831    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909835    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909837    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909840    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909843    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909845    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:32:33.914592 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.909848    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.909853    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910015    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910021    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910024    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910027    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910030    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910033    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910035    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910038    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910040    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910043    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910045    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910055    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910058    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 15:32:33.914963 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910061    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910063    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910066    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910069    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910071    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910074    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910077    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910079    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910082    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910085    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910088    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910090    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910094    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910097    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910099    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910102    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910114    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910118    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910122    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 15:32:33.915356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910125    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910128    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910131    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910134    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910137    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910140    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910143    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910145    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910148    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910151    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910153    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910156    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910164    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910167    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910171    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910174    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910177    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910180    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910183    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910186    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 15:32:33.915836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910192    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910195    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910199    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910202    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910205    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910208    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910211    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910214    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910217    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910220    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910223    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910226    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910229    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910231    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910234    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910237    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910239    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910242    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910244    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910247    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 15:32:33.916338 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910250    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910252    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910255    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910257    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910260    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910268    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910271    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910274    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910276    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910279    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910282    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910300    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910303    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910306    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910309    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910312    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910315    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910317    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910320    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 15:32:33.916878 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910322    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910326    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910330    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910333    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910336    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910339    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910341    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:33.910344    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.910349    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.910488    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.914640    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.915595    2563 server.go:1019] "Starting client certificate rotation"
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.915735    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:32:33.917429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.915776    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 15:32:33.922035 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.922009    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:32:33.925144 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.925113    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 15:32:33.934560 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.934538    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 08 15:32:33.939906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.939887    2563 log.go:25] "Validated CRI v1 image API"
Sep 08 15:32:33.940808 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.940778    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 15:32:33.946377 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.946355    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cd47884b-8d72-4af8-9951-c7bd83a154a9:/dev/nvme0n1p3 ddd2f59c-9877-43b9-95f1-2e4cd8086332:/dev/nvme0n1p4]
Sep 08 15:32:33.946452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.946376    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 15:32:33.948677 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.948654    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:32:33.952160 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.952015    2563 manager.go:216] Machine: {Timestamp:2026-09-08 15:32:33.950682894 +0000 UTC m=+0.393875114 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088142 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2466e71f758607b1df1427f05cb219 SystemUUID:ec2466e7-1f75-8607-b1df-1427f05cb219 BootID:6ea2ac04-f77c-4c9f-945f-cacaf4cee516 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:51:2b:52:fe:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:51:2b:52:fe:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:fd:27:f3:a3:1e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 15:32:33.952160 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.952155    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 15:32:33.952343 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.952329    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 15:32:33.954498 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.954462    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 15:32:33.954668 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.954500    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-240.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 15:32:33.956067 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.956047    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 15:32:33.956126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.956068    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 15:32:33.956126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.956078    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:32:33.956896 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.956880    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 15:32:33.958642 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.958625    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 15:32:33.963957 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.963935    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:32:33.964102 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.964091    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 15:32:33.964162 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.964133    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 15:32:33.964162 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.964147    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 15:32:33.964162 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.964158    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 15:32:33.964276 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.964185    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 08 15:32:33.964276 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.964195    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 15:32:33.965436 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.965389    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:32:33.967248 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.967226    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 15:32:33.970876 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.970838    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 15:32:33.973165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.973141    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rtg8x"
Sep 08 15:32:33.974133 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974115    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 15:32:33.974236 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974147    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 15:32:33.974453 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974439    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 15:32:33.974508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974460    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 15:32:33.974508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974469    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 15:32:33.974508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974478    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 15:32:33.974508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974489    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 15:32:33.974508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974497    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 15:32:33.975011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.974998    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 15:32:33.975011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.975012    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 15:32:33.975068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.975021    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 15:32:33.975068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.975027    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 15:32:33.975068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.975038    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 15:32:33.975068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.975046    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 15:32:33.976845 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.976834    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 15:32:33.976880 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.976848    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 15:32:33.978371 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:33.978341    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 15:32:33.978445 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:33.978346    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-240.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 15:32:33.979953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.979934    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rtg8x"
Sep 08 15:32:33.981656 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.981641    2563 server.go:1268] "Started kubelet"
Sep 08 15:32:33.981791 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.981737    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 15:32:33.981828 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.981755    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 15:32:33.981856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.981849    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 15:32:33.982650 ip-10-0-135-240 systemd[1]: Started Kubernetes Kubelet.
Sep 08 15:32:33.983173 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.982958    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 15:32:33.986617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.986596    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 15:32:33.988515 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.988491    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-240.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 15:32:33.988661 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:33.988633    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 15:32:33.989631 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:33.988518    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-240.ec2.internal.18d3624ba25286c4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-240.ec2.internal,UID:ip-10-0-135-240.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-240.ec2.internal,},FirstTimestamp:2026-09-08 15:32:33.981605572 +0000 UTC m=+0.424797771,LastTimestamp:2026-09-08 15:32:33.981605572 +0000 UTC m=+0.424797771,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-240.ec2.internal,}"
Sep 08 15:32:33.990431 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.990417    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 15:32:33.990488 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.990428    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 15:32:33.990535 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:33.990504    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:33.991216 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991196    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 15:32:33.991316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991203    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 15:32:33.991316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991240    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 15:32:33.991428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991357    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 15:32:33.991428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991367    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 15:32:33.991513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991455    2563 factory.go:55] Registering systemd factory
Sep 08 15:32:33.991513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991471    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 08 15:32:33.991730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991713    2563 factory.go:153] Registering CRI-O factory
Sep 08 15:32:33.991780 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991733    2563 factory.go:223] Registration of the crio container factory successfully
Sep 08 15:32:33.991829 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991809    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 15:32:33.991866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991847    2563 factory.go:103] Registering Raw factory
Sep 08 15:32:33.991866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.991859    2563 manager.go:1196] Started watching for new ooms in manager
Sep 08 15:32:33.992456 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.992428    2563 manager.go:349] Starting recovery of all containers
Sep 08 15:32:33.995115 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:33.995093    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:32:33.999104 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:33.999075    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.006213 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.006189    2563 manager.go:354] Recovery completed
Sep 08 15:32:34.010953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.010934    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.012844 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.012822    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.012939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.012857    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.012939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.012869    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.013461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.013447    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 15:32:34.013461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.013457    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.013461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.013461    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 15:32:34.013612 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.013483    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 15:32:34.014616 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.014599    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.014684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.014634    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.014684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.014649    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.015584 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.015570    2563 policy_none.go:49] "None policy: Start"
Sep 08 15:32:34.015644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.015588    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 15:32:34.015644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.015598    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 15:32:34.017464 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.017453    2563 policy_none.go:47] "Start"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060138    2563 manager.go:337] "Starting Device Plugin manager"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.060184    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060202    2563 server.go:87] "Starting device plugin registration server"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060554    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060566    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060678    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060773    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.060783    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.061274    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 15:32:34.071680 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.061345    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.092449 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.092415    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 15:32:34.093709 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.093691    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 15:32:34.093801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.093754    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 15:32:34.093801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.093790    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 15:32:34.093896 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.093844    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 15:32:34.096362 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.096342    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:32:34.161348 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.161228    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.162495 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.162479    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.162567 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.162509    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.162567 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.162520    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.162567 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.162545    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.170796 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.170775    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.170867 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.170806    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-240.ec2.internal\": node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.186369 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.186336    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.194600 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.194568    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"]
Sep 08 15:32:34.194725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.194690    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.196465 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.196446    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.196584 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.196484    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.196584 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.196519    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.198841 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.198821    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.198964 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.198946    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.199017 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.198993    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.199740 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.199722    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.199832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.199750    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.199832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.199760    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.199832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.199722    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.199832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.199823    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.199832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.199835    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.202127 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.202109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.202189 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.202148    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 15:32:34.202993 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.202976    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 15:32:34.203091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.203008    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 15:32:34.203091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.203021    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeHasSufficientPID"
Sep 08 15:32:34.225302 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.225265    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.229878 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.229859    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-240.ec2.internal\" not found" node="ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.286480 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.286441    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.292738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.292716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27659e071ebcf76aabb813758c1a03eb-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"27659e071ebcf76aabb813758c1a03eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.292811 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.292744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/55bd377e98a5e435a35cefc9a800c096-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"55bd377e98a5e435a35cefc9a800c096\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.292811 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.292784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55bd377e98a5e435a35cefc9a800c096-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"55bd377e98a5e435a35cefc9a800c096\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.387364 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.387330    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.393739 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.393709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/55bd377e98a5e435a35cefc9a800c096-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"55bd377e98a5e435a35cefc9a800c096\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.393820 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.393746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55bd377e98a5e435a35cefc9a800c096-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"55bd377e98a5e435a35cefc9a800c096\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.393820 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.393763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27659e071ebcf76aabb813758c1a03eb-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"27659e071ebcf76aabb813758c1a03eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.393893 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.393813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/55bd377e98a5e435a35cefc9a800c096-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"55bd377e98a5e435a35cefc9a800c096\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.393893 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.393871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/27659e071ebcf76aabb813758c1a03eb-config\") pod \"kube-apiserver-proxy-ip-10-0-135-240.ec2.internal\" (UID: \"27659e071ebcf76aabb813758c1a03eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.393964 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.393900    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55bd377e98a5e435a35cefc9a800c096-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal\" (UID: \"55bd377e98a5e435a35cefc9a800c096\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.488219 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.488148    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.528679 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.528639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.533527 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.533399    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:34.589157 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.589112    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.689630 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.689595    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.790142 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.790062    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.890602 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.890559    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:34.916020 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.915975    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 15:32:34.916160 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.916142    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:32:34.916204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.916171    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 15:32:34.988640 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.988582    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 15:27:34 +0000 UTC" deadline="2028-02-18 23:22:50.765587943 +0000 UTC"
Sep 08 15:32:34.988640 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:34.988625    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12679h50m15.776965936s"
Sep 08 15:32:34.991338 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:34.991318    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:35.091908 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:35.091885    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-240.ec2.internal\" not found"
Sep 08 15:32:35.100392 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.100363    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:32:35.191469 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.191428    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:35.203832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.203810    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:32:35.205570 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.205548    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal"
Sep 08 15:32:35.208058 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:35.208023    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bd377e98a5e435a35cefc9a800c096.slice/crio-cf30eecf5f516282aa540d9153dd3e3554dd1fba371df34c6eeaae3dbe64a4b0 WatchSource:0}: Error finding container cf30eecf5f516282aa540d9153dd3e3554dd1fba371df34c6eeaae3dbe64a4b0: Status 404 returned error can't find the container with id cf30eecf5f516282aa540d9153dd3e3554dd1fba371df34c6eeaae3dbe64a4b0
Sep 08 15:32:35.212410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.212392    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 15:32:35.215016 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.215000    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:32:35.416777 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.416684    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:32:35.911658 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.911569    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:32:35.918442 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.918414    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 15:32:35.966091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.965822    2563 apiserver.go:52] "Watching apiserver"
Sep 08 15:32:35.972065 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.972036    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 15:32:35.972783 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.972560    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-4xdfr","kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal","openshift-dns/node-resolver-qldnr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal","openshift-multus/multus-s9wdl","openshift-multus/network-metrics-daemon-dxkzl","openshift-network-diagnostics/network-check-target-gl426","openshift-network-operator/iptables-alerter-mtxpj","kube-system/global-pull-secret-syncer-pbvp6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7","openshift-cluster-node-tuning-operator/tuned-qnrkp","openshift-image-registry/node-ca-v2cz2","openshift-multus/multus-additional-cni-plugins-mxdzc","openshift-ovn-kubernetes/ovnkube-node-ktl7v"]
Sep 08 15:32:35.975196 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.975173    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:35.977550 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.977519    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 15:32:35.977698 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.977675    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 15:32:35.977880 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.977762    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fjjbn\""
Sep 08 15:32:35.977945 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.977923    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:35.977995 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.977975    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 15:32:35.979870 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.979844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:35.980942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.980244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 15:32:35.980942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.980312    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 15:32:35.980942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.980695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-smvfq\""
Sep 08 15:32:35.980942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.980736    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 15:32:35.980942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.980815    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 15:32:35.981242 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.981059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 15:32:35.981942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.981919    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 15:32:35.982247 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.982232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:35.982636 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.982370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-89p88\""
Sep 08 15:32:35.982882 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.982864    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 15:32:35.984851 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.984607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.985749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.985835    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.985760    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.985906    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.985978    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.985836    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2fdcv\""
Sep 08 15:32:35.986541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.986100    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 15:32:35.986876 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.986560    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 15:32:35.986876 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.986802    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vzwmq\""
Sep 08 15:32:35.987930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.987908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:35.988052 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:35.988026    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:35.991389 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.991355    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 15:32:35.993885 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.993564    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:35.993885 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:35.993656    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:35.996174 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.996148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:35.996304 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:35.996225    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:35.998842 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:35.998784    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.001246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.001218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.002503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002478    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 15:32:36.002608 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-k8s-cni-cncf-io\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.002608 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-cni-bin\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.002730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-kubelet\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.002730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-run-netns\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.002730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-kubelet\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.002730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.002730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66hp7\" (UniqueName: \"kubernetes.io/projected/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-kube-api-access-66hp7\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-etc-kubernetes\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-etc-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-cnibin\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-multus-certs\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/13af96a0-8a7f-479c-887d-54f937793c41-dbus\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-socket-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-run-ovn-kubernetes\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.002967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-hostroot\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.002983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de883f2-c452-4202-b140-874f16695b98-multus-daemon-config\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-system-cni-dir\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2r7s\" (UniqueName: \"kubernetes.io/projected/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-kube-api-access-h2r7s\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-node-log\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-ovnkube-script-lib\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-system-cni-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-var-lib-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-cni-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-device-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.003500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5vpc\" (UniqueName: \"kubernetes.io/projected/3de883f2-c452-4202-b140-874f16695b98-kube-api-access-b5vpc\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-cnibin\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-hosts-file\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d91aa006-adab-4913-97fd-cbd01c28f594-ovn-node-metrics-cert\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-socket-dir-parent\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm9fg\" (UniqueName: \"kubernetes.io/projected/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-kube-api-access-cm9fg\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wglwm\" (UniqueName: \"kubernetes.io/projected/847c4a7e-f4af-460b-9747-b082b1b323ec-kube-api-access-wglwm\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-tmp-dir\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-env-overrides\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-cni-multus\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-systemd-units\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.004179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-cni-bin\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.003981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de883f2-c452-4202-b140-874f16695b98-cni-binary-copy\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/13af96a0-8a7f-479c-887d-54f937793c41-kubelet-config\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-sys-fs\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-systemd\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004056    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-g45lg\""
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-ovn\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-slash\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-log-socket\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-cni-netd\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-ovnkube-config\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-netns\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-conf-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004321    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-registration-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dksnh\" (UniqueName: \"kubernetes.io/projected/d91aa006-adab-4913-97fd-cbd01c28f594-kube-api-access-dksnh\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004423    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004600    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sb9v8\""
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004642    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 15:32:36.005033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.004748    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.006012 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.005999    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 15:32:36.006642 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.006621    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.007697 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.007679    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:32:36.007990 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.007972    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 15:32:36.008385 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.008358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rfmfd\""
Sep 08 15:32:36.009793 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.009540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7476w\""
Sep 08 15:32:36.009793 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.009607    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 15:32:36.009950 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.009830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 15:32:36.010869 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.010847    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 15:32:36.035256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.035225    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vj5zt"
Sep 08 15:32:36.043029 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.043002    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vj5zt"
Sep 08 15:32:36.092357 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.092318    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 15:32:36.101037 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.100980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" event={"ID":"27659e071ebcf76aabb813758c1a03eb","Type":"ContainerStarted","Data":"e602147dd60d33909f285fe20a9ec019d37c7d16f37c6227595002089345f5e3"}
Sep 08 15:32:36.102336 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.102280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"55bd377e98a5e435a35cefc9a800c096","Type":"ContainerStarted","Data":"cf30eecf5f516282aa540d9153dd3e3554dd1fba371df34c6eeaae3dbe64a4b0"}
Sep 08 15:32:36.104772 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66hp7\" (UniqueName: \"kubernetes.io/projected/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-kube-api-access-66hp7\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-etc-kubernetes\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-sys\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-etc-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.104966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-cnibin\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-etc-kubernetes\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105019    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-etc-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-multus-certs\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.104972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-multus-certs\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-iptables-alerter-script\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-cnibin\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzr89\" (UniqueName: \"kubernetes.io/projected/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-kube-api-access-tzr89\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/13af96a0-8a7f-479c-887d-54f937793c41-dbus\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-socket-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.105257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-run-ovn-kubernetes\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-hostroot\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de883f2-c452-4202-b140-874f16695b98-multus-daemon-config\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-systemd\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-run-ovn-kubernetes\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-hostroot\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/13af96a0-8a7f-479c-887d-54f937793c41-dbus\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f439e87-2fcb-430d-960b-eb74a8e5efdd-serviceca\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105440    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-socket-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-system-cni-dir\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2r7s\" (UniqueName: \"kubernetes.io/projected/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-kube-api-access-h2r7s\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-node-log\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-system-cni-dir\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105587    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-node-log\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-ovnkube-script-lib\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-system-cni-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.105836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/371a2499-4888-48a7-99ea-2307c24cdad6-konnectivity-ca\") pod \"konnectivity-agent-4xdfr\" (UID: \"371a2499-4888-48a7-99ea-2307c24cdad6\") " pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-var-lib-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-cni-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-system-cni-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-host-slash\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-device-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-var-lib-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5vpc\" (UniqueName: \"kubernetes.io/projected/3de883f2-c452-4202-b140-874f16695b98-kube-api-access-b5vpc\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.105884    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-run\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-cnibin\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-hosts-file\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.105983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d91aa006-adab-4913-97fd-cbd01c28f594-ovn-node-metrics-cert\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-socket-dir-parent\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.106723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3de883f2-c452-4202-b140-874f16695b98-multus-daemon-config\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-hosts-file\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-kubernetes\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-lib-modules\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-cni-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-device-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106159    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/847c4a7e-f4af-460b-9747-b082b1b323ec-cnibin\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.106184    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:32:36.606158432 +0000 UTC m=+3.049350626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-openvswitch\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm9fg\" (UniqueName: \"kubernetes.io/projected/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-kube-api-access-cm9fg\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-ovnkube-script-lib\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wglwm\" (UniqueName: \"kubernetes.io/projected/847c4a7e-f4af-460b-9747-b082b1b323ec-kube-api-access-wglwm\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106269    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-var-lib-kubelet\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.107787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/371a2499-4888-48a7-99ea-2307c24cdad6-agent-certs\") pod \"konnectivity-agent-4xdfr\" (UID: \"371a2499-4888-48a7-99ea-2307c24cdad6\") " pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysconfig\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99161744-d411-4b90-8108-2779f657486a-tmp\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-tmp-dir\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-socket-dir-parent\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-env-overrides\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.106527    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-cni-multus\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.106598    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:32:36.606577871 +0000 UTC m=+3.049770081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-cni-multus\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-systemd-units\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106626    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-systemd-units\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-cni-bin\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de883f2-c452-4202-b140-874f16695b98-cni-binary-copy\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.108478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-cni-bin\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrdx7\" (UniqueName: \"kubernetes.io/projected/3f439e87-2fcb-430d-960b-eb74a8e5efdd-kube-api-access-vrdx7\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/13af96a0-8a7f-479c-887d-54f937793c41-kubelet-config\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-sys-fs\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106788    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/13af96a0-8a7f-479c-887d-54f937793c41-kubelet-config\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106788    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-tmp-dir\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-systemd\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-ovn\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysctl-conf\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99161744-d411-4b90-8108-2779f657486a-etc-tuned\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-systemd\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-env-overrides\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.106982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-slash\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-log-socket\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-sys-fs\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.109294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-run-ovn\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-cni-netd\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-ovnkube-config\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-slash\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-netns\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-conf-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-log-socket\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-modprobe-d\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-cni-netd\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-netns\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107184    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-multus-conf-dir\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-registration-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3de883f2-c452-4202-b140-874f16695b98-cni-binary-copy\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dksnh\" (UniqueName: \"kubernetes.io/projected/d91aa006-adab-4913-97fd-cbd01c28f594-kube-api-access-dksnh\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-k8s-cni-cncf-io\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-cni-bin\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-registration-dir\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-host\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.109929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-run-k8s-cni-cncf-io\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx9wv\" (UniqueName: \"kubernetes.io/projected/99161744-d411-4b90-8108-2779f657486a-kube-api-access-bx9wv\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f439e87-2fcb-430d-960b-eb74a8e5efdd-host\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-cni-bin\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-kubelet\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-run-netns\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-kubelet\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-kubelet\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysctl-d\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/847c4a7e-f4af-460b-9747-b082b1b323ec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d91aa006-adab-4913-97fd-cbd01c28f594-host-run-netns\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3de883f2-c452-4202-b140-874f16695b98-host-var-lib-kubelet\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.110781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.107648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d91aa006-adab-4913-97fd-cbd01c28f594-ovnkube-config\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.111450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.111058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d91aa006-adab-4913-97fd-cbd01c28f594-ovn-node-metrics-cert\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.128715 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.128678    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66hp7\" (UniqueName: \"kubernetes.io/projected/a82652f7-3dc5-4132-9dd6-02e12e6a95cd-kube-api-access-66hp7\") pod \"node-resolver-qldnr\" (UID: \"a82652f7-3dc5-4132-9dd6-02e12e6a95cd\") " pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.135963 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.135926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dksnh\" (UniqueName: \"kubernetes.io/projected/d91aa006-adab-4913-97fd-cbd01c28f594-kube-api-access-dksnh\") pod \"ovnkube-node-ktl7v\" (UID: \"d91aa006-adab-4913-97fd-cbd01c28f594\") " pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.136785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.136730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm9fg\" (UniqueName: \"kubernetes.io/projected/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-kube-api-access-cm9fg\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:36.138168 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.138111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5vpc\" (UniqueName: \"kubernetes.io/projected/3de883f2-c452-4202-b140-874f16695b98-kube-api-access-b5vpc\") pod \"multus-s9wdl\" (UID: \"3de883f2-c452-4202-b140-874f16695b98\") " pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.139604 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.139160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2r7s\" (UniqueName: \"kubernetes.io/projected/da9eda6e-52f9-470c-a05a-e027f2cdc9ca-kube-api-access-h2r7s\") pod \"aws-ebs-csi-driver-node-p4zs7\" (UID: \"da9eda6e-52f9-470c-a05a-e027f2cdc9ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.139723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.139692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wglwm\" (UniqueName: \"kubernetes.io/projected/847c4a7e-f4af-460b-9747-b082b1b323ec-kube-api-access-wglwm\") pod \"multus-additional-cni-plugins-mxdzc\" (UID: \"847c4a7e-f4af-460b-9747-b082b1b323ec\") " pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.208951 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.208860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-systemd\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.208951 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.208905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f439e87-2fcb-430d-960b-eb74a8e5efdd-serviceca\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.209190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-systemd\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:36.209190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/371a2499-4888-48a7-99ea-2307c24cdad6-konnectivity-ca\") pod \"konnectivity-agent-4xdfr\" (UID: \"371a2499-4888-48a7-99ea-2307c24cdad6\") " pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.209190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-host-slash\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-run\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-kubernetes\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-lib-modules\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-var-lib-kubelet\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/371a2499-4888-48a7-99ea-2307c24cdad6-agent-certs\") pod \"konnectivity-agent-4xdfr\" (UID: \"371a2499-4888-48a7-99ea-2307c24cdad6\") " pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysconfig\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99161744-d411-4b90-8108-2779f657486a-tmp\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrdx7\" (UniqueName: \"kubernetes.io/projected/3f439e87-2fcb-430d-960b-eb74a8e5efdd-kube-api-access-vrdx7\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysctl-conf\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3f439e87-2fcb-430d-960b-eb74a8e5efdd-serviceca\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99161744-d411-4b90-8108-2779f657486a-etc-tuned\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-modprobe-d\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysconfig\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-host\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-run\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-lib-modules\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx9wv\" (UniqueName: \"kubernetes.io/projected/99161744-d411-4b90-8108-2779f657486a-kube-api-access-bx9wv\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-var-lib-kubelet\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f439e87-2fcb-430d-960b-eb74a8e5efdd-host\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-kubernetes\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/371a2499-4888-48a7-99ea-2307c24cdad6-konnectivity-ca\") pod \"konnectivity-agent-4xdfr\" (UID: \"371a2499-4888-48a7-99ea-2307c24cdad6\") " pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-host-slash\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysctl-d\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.209781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f439e87-2fcb-430d-960b-eb74a8e5efdd-host\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.210599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-sys\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.210599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-sys\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.210599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysctl-conf\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.210599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209956    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-iptables-alerter-script\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.210599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.209995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr89\" (UniqueName: \"kubernetes.io/projected/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-kube-api-access-tzr89\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.210599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.210577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-iptables-alerter-script\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.210879 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.210807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-host\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.210933 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.210889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-modprobe-d\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.210973 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.210937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/99161744-d411-4b90-8108-2779f657486a-etc-sysctl-d\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.212504 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.212467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99161744-d411-4b90-8108-2779f657486a-tmp\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.212873 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.212849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/371a2499-4888-48a7-99ea-2307c24cdad6-agent-certs\") pod \"konnectivity-agent-4xdfr\" (UID: \"371a2499-4888-48a7-99ea-2307c24cdad6\") " pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.212982 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.212852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/99161744-d411-4b90-8108-2779f657486a-etc-tuned\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.216303 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.216259    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:32:36.216303 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.216284    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:32:36.216303 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.216299    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:36.216547 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.216388    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:32:36.716348975 +0000 UTC m=+3.159541181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:36.218481 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.218456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrdx7\" (UniqueName: \"kubernetes.io/projected/3f439e87-2fcb-430d-960b-eb74a8e5efdd-kube-api-access-vrdx7\") pod \"node-ca-v2cz2\" (UID: \"3f439e87-2fcb-430d-960b-eb74a8e5efdd\") " pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.219261 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.219236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzr89\" (UniqueName: \"kubernetes.io/projected/a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1-kube-api-access-tzr89\") pod \"iptables-alerter-mtxpj\" (UID: \"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1\") " pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.219379 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.219309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx9wv\" (UniqueName: \"kubernetes.io/projected/99161744-d411-4b90-8108-2779f657486a-kube-api-access-bx9wv\") pod \"tuned-qnrkp\" (UID: \"99161744-d411-4b90-8108-2779f657486a\") " pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.290353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.290319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7"
Sep 08 15:32:36.297956 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.297924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mxdzc"
Sep 08 15:32:36.302480 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.302447    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9eda6e_52f9_470c_a05a_e027f2cdc9ca.slice/crio-010ae63e7f924f104d04e6a6f6d7ca629998966c2e4e4218a37bc18e9792c786 WatchSource:0}: Error finding container 010ae63e7f924f104d04e6a6f6d7ca629998966c2e4e4218a37bc18e9792c786: Status 404 returned error can't find the container with id 010ae63e7f924f104d04e6a6f6d7ca629998966c2e4e4218a37bc18e9792c786
Sep 08 15:32:36.308203 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.308177    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847c4a7e_f4af_460b_9747_b082b1b323ec.slice/crio-743874749d68f0a5d09ae16f3ff90e6ff9fa854cb8ae3bda763ac40e4eb1610a WatchSource:0}: Error finding container 743874749d68f0a5d09ae16f3ff90e6ff9fa854cb8ae3bda763ac40e4eb1610a: Status 404 returned error can't find the container with id 743874749d68f0a5d09ae16f3ff90e6ff9fa854cb8ae3bda763ac40e4eb1610a
Sep 08 15:32:36.309437 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.309412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qldnr"
Sep 08 15:32:36.315175 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.315102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:32:36.321555 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.321519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s9wdl"
Sep 08 15:32:36.323440 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.323402    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82652f7_3dc5_4132_9dd6_02e12e6a95cd.slice/crio-b481183d9226a1839018a0564add9aaacb7110d528d862f8b59c35913541708b WatchSource:0}: Error finding container b481183d9226a1839018a0564add9aaacb7110d528d862f8b59c35913541708b: Status 404 returned error can't find the container with id b481183d9226a1839018a0564add9aaacb7110d528d862f8b59c35913541708b
Sep 08 15:32:36.327404 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.327378    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91aa006_adab_4913_97fd_cbd01c28f594.slice/crio-b24dd802f48bab1fccc98af66ff57fca471b2f42e6e3eab35ab88b12999fff94 WatchSource:0}: Error finding container b24dd802f48bab1fccc98af66ff57fca471b2f42e6e3eab35ab88b12999fff94: Status 404 returned error can't find the container with id b24dd802f48bab1fccc98af66ff57fca471b2f42e6e3eab35ab88b12999fff94
Sep 08 15:32:36.330108 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.330076    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mtxpj"
Sep 08 15:32:36.335329 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.335292    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3de883f2_c452_4202_b140_874f16695b98.slice/crio-5cccfcc8de9763d6838ace5cd011fd8516b7976333ca1b091a460437876ea5e7 WatchSource:0}: Error finding container 5cccfcc8de9763d6838ace5cd011fd8516b7976333ca1b091a460437876ea5e7: Status 404 returned error can't find the container with id 5cccfcc8de9763d6838ace5cd011fd8516b7976333ca1b091a460437876ea5e7
Sep 08 15:32:36.338645 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.338619    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:36.343426 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.343393    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13ff2a5_6eba_4fb7_b1a6_2e77810b94f1.slice/crio-e36cce1b981fa32c47aed5e4a8d1b3059b90e6ab0903e8df1c55540a53734b32 WatchSource:0}: Error finding container e36cce1b981fa32c47aed5e4a8d1b3059b90e6ab0903e8df1c55540a53734b32: Status 404 returned error can't find the container with id e36cce1b981fa32c47aed5e4a8d1b3059b90e6ab0903e8df1c55540a53734b32
Sep 08 15:32:36.348092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.348065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnrkp"
Sep 08 15:32:36.349117 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:36.349082    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod371a2499_4888_48a7_99ea_2307c24cdad6.slice/crio-432e508e088d289061d2f46520465f246cef32056d492abdca8a3e066b70f2a2 WatchSource:0}: Error finding container 432e508e088d289061d2f46520465f246cef32056d492abdca8a3e066b70f2a2: Status 404 returned error can't find the container with id 432e508e088d289061d2f46520465f246cef32056d492abdca8a3e066b70f2a2
Sep 08 15:32:36.362721 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.362689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v2cz2"
Sep 08 15:32:36.613845 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.613735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:36.613845 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.613799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:36.614088 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.613932    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:36.614088 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.614019    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:32:37.613998665 +0000 UTC m=+4.057190869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:36.614088 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.613936    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:36.614245 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.614095    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:32:37.614078117 +0000 UTC m=+4.057270308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:36.815710 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:36.815662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:36.815905 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.815880    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:32:36.815965 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.815911    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:32:36.815965 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.815927    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:36.816056 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:36.816001    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:32:37.815977436 +0000 UTC m=+4.259169631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:37.046374 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.046243    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:27:36 +0000 UTC" deadline="2028-05-01 16:27:02.596736935 +0000 UTC"
Sep 08 15:32:37.046374 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.046312    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14424h54m25.550428827s"
Sep 08 15:32:37.070580 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:32:37.070539    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f439e87_2fcb_430d_960b_eb74a8e5efdd.slice/crio-bc029dbf38a22871c6f5879f33601885da841fad24e6dc5e3cc6042ce56f8650 WatchSource:0}: Error finding container bc029dbf38a22871c6f5879f33601885da841fad24e6dc5e3cc6042ce56f8650: Status 404 returned error can't find the container with id bc029dbf38a22871c6f5879f33601885da841fad24e6dc5e3cc6042ce56f8650
Sep 08 15:32:37.095012 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.094979    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:37.095188 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.095102    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:37.107869 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.107831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerStarted","Data":"743874749d68f0a5d09ae16f3ff90e6ff9fa854cb8ae3bda763ac40e4eb1610a"}
Sep 08 15:32:37.109025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.108995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v2cz2" event={"ID":"3f439e87-2fcb-430d-960b-eb74a8e5efdd","Type":"ContainerStarted","Data":"bc029dbf38a22871c6f5879f33601885da841fad24e6dc5e3cc6042ce56f8650"}
Sep 08 15:32:37.110308 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.110265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnrkp" event={"ID":"99161744-d411-4b90-8108-2779f657486a","Type":"ContainerStarted","Data":"4d21a54f2a31b6c0821e9ffcbc35d1a18c80e27aaad14f5f88a0418a56ff6495"}
Sep 08 15:32:37.111410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.111387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4xdfr" event={"ID":"371a2499-4888-48a7-99ea-2307c24cdad6","Type":"ContainerStarted","Data":"432e508e088d289061d2f46520465f246cef32056d492abdca8a3e066b70f2a2"}
Sep 08 15:32:37.112474 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.112450    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mtxpj" event={"ID":"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1","Type":"ContainerStarted","Data":"e36cce1b981fa32c47aed5e4a8d1b3059b90e6ab0903e8df1c55540a53734b32"}
Sep 08 15:32:37.113774 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.113753    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9wdl" event={"ID":"3de883f2-c452-4202-b140-874f16695b98","Type":"ContainerStarted","Data":"5cccfcc8de9763d6838ace5cd011fd8516b7976333ca1b091a460437876ea5e7"}
Sep 08 15:32:37.114906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.114883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qldnr" event={"ID":"a82652f7-3dc5-4132-9dd6-02e12e6a95cd","Type":"ContainerStarted","Data":"b481183d9226a1839018a0564add9aaacb7110d528d862f8b59c35913541708b"}
Sep 08 15:32:37.115983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.115955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7" event={"ID":"da9eda6e-52f9-470c-a05a-e027f2cdc9ca","Type":"ContainerStarted","Data":"010ae63e7f924f104d04e6a6f6d7ca629998966c2e4e4218a37bc18e9792c786"}
Sep 08 15:32:37.117413 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.117390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"b24dd802f48bab1fccc98af66ff57fca471b2f42e6e3eab35ab88b12999fff94"}
Sep 08 15:32:37.623815 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.623746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:37.623815 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.623812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:37.624073 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.623949    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:37.624073 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.624017    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:32:39.623997004 +0000 UTC m=+6.067189195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:37.624411 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.624362    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:37.624521 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.624442    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:32:39.624421308 +0000 UTC m=+6.067613514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:37.825703 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:37.824959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:37.825703 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.825229    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:32:37.825703 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.825255    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:32:37.825703 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.825272    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:37.825703 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:37.825354    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:32:39.825319748 +0000 UTC m=+6.268511942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:38.047303 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:38.046948    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 15:27:36 +0000 UTC" deadline="2028-05-04 07:32:45.60134127 +0000 UTC"
Sep 08 15:32:38.047303 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:38.046993    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14488h0m7.554353431s"
Sep 08 15:32:38.101617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:38.101585    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:38.101779 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:38.101744    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:38.102208 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:38.102174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:38.102313 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:38.102277    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:38.132996 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:38.132008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" event={"ID":"27659e071ebcf76aabb813758c1a03eb","Type":"ContainerStarted","Data":"8a892ba5a160511e0626540af9a64c873d97c75e88ae8b7534133124eecac696"}
Sep 08 15:32:38.147628 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:38.147560    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-240.ec2.internal" podStartSLOduration=2.147537945 podStartE2EDuration="2.147537945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:38.14712544 +0000 UTC m=+4.590317655" watchObservedRunningTime="2026-09-08 15:32:38.147537945 +0000 UTC m=+4.590730159"
Sep 08 15:32:39.094756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:39.094716    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:39.095353 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.094863    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:39.645774 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:39.645717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:39.645997 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:39.645822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:39.650186 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.649561    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:39.650186 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.649678    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:32:43.649655157 +0000 UTC m=+10.092847363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:39.653680 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.650603    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:39.653680 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.650685    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:32:43.650664116 +0000 UTC m=+10.093856323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:39.848213 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:39.848164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:39.848457 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.848423    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:32:39.848457 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.848449    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:32:39.848579 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.848463    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:39.848579 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:39.848533    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:32:43.848510457 +0000 UTC m=+10.291702652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:40.094367 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:40.094248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:40.094548 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:40.094265    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:40.094548 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:40.094433    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:40.094548 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:40.094510    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:41.095554 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:41.095031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:41.095554 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:41.095199    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:42.094473 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:42.094429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:42.094661 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:42.094608    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:42.095097 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:42.095074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:42.095195 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:42.095180    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:43.095010 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:43.094699    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:43.095560 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.095141    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:43.684148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:43.684036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:43.684148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:43.684153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:43.684465 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.684237    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:43.684465 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.684291    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:43.684465 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.684351    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:32:51.684326244 +0000 UTC m=+18.127518449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:43.684465 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.684378    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:32:51.684366232 +0000 UTC m=+18.127558427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:43.886707 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:43.886665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:43.886915 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.886843    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:32:43.886915 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.886865    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:32:43.886915 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.886878    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:43.887078 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:43.886941    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:32:51.886918308 +0000 UTC m=+18.330110498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:44.095611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:44.095519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:44.096094 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:44.095674    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:44.096173 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:44.096101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:44.096228 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:44.096201    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:45.094615 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:45.094574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:45.094831 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:45.094721    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:46.094886 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:46.094848    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:46.095427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:46.094860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:46.095427 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:46.095007    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:46.095427 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:46.095095    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:47.094577 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:47.094530    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:47.094781 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:47.094665    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:48.094353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:48.094317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:48.094736 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:48.094317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:48.094736 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:48.094445    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:48.094736 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:48.094509    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:49.094189 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:49.094102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:49.094384 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:49.094224    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:50.094250 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:50.094189    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:50.094450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:50.094189    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:50.094450 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:50.094363    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:50.094450 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:50.094412    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:51.094980 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:51.094945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:51.095471 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.095077    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:51.752393 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:51.752355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:51.752565 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:51.752405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:51.752565 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.752517    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:51.752565 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.752521    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:51.752728 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.752576    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:33:07.752562104 +0000 UTC m=+34.195754290 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:32:51.752728 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.752589    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:07.752583573 +0000 UTC m=+34.195775760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:32:51.954012 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:51.953968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:51.954218 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.954169    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:32:51.954218 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.954199    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:32:51.954218 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.954211    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:51.954388 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:51.954300    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:33:07.954262105 +0000 UTC m=+34.397454309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:32:52.094990 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:52.094913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:52.095145 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:52.094913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:52.095145 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:52.095039    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:52.095145 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:52.095110    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:53.094883 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:53.094850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:53.095054 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:53.094976    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:54.096772 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:54.096735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:54.097539 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:54.097500    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:54.098108 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:54.098086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:54.098338 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:54.098316    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:55.095080 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:55.095042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:55.095253 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:55.095169    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:56.094716 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:56.094678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:56.095125 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:56.094680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:56.095125 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:56.094834    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:56.095125 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:56.094870    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:57.094710 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.094530    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:57.094887 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:57.094788    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:57.174977 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.174944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v2cz2" event={"ID":"3f439e87-2fcb-430d-960b-eb74a8e5efdd","Type":"ContainerStarted","Data":"abcacf8a69138c9a2d0f4e1a40fdd180e658e64d8d16f7058957a28df107a8ed"}
Sep 08 15:32:57.176458 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.176428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnrkp" event={"ID":"99161744-d411-4b90-8108-2779f657486a","Type":"ContainerStarted","Data":"c64fb0e830ac6d16d11281578659f83db5273565a5b5678c54edadbabe807a87"}
Sep 08 15:32:57.177796 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.177774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4xdfr" event={"ID":"371a2499-4888-48a7-99ea-2307c24cdad6","Type":"ContainerStarted","Data":"637b66b49647a24a41b0b1b1b54ee39d1f51b89cfa538519f1aaaef0e54dc6ee"}
Sep 08 15:32:57.179204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.179173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s9wdl" event={"ID":"3de883f2-c452-4202-b140-874f16695b98","Type":"ContainerStarted","Data":"94f94fdb3e92b30a1e7695c35097a0eb7210db67a291ff3d27f637d727e323a0"}
Sep 08 15:32:57.180509 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.180488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qldnr" event={"ID":"a82652f7-3dc5-4132-9dd6-02e12e6a95cd","Type":"ContainerStarted","Data":"3ad50ef1976a8262c2ee4a96b6740d59526c8c7128951b03a74632782b780e92"}
Sep 08 15:32:57.181995 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.181974    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7" event={"ID":"da9eda6e-52f9-470c-a05a-e027f2cdc9ca","Type":"ContainerStarted","Data":"e883ceaf133e3b717b5ded1d0133359850e2b0190cdd5247681ecd176c713f3a"}
Sep 08 15:32:57.183384 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.183363    2563 generic.go:356] "Generic (PLEG): container finished" podID="55bd377e98a5e435a35cefc9a800c096" containerID="12d3a00a0c41581b4fee8c90cc0beae7058a22bc07cc0c8b9244431ffba1190b" exitCode=0
Sep 08 15:32:57.183462 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.183440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"55bd377e98a5e435a35cefc9a800c096","Type":"ContainerDied","Data":"12d3a00a0c41581b4fee8c90cc0beae7058a22bc07cc0c8b9244431ffba1190b"}
Sep 08 15:32:57.186206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.186185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"3d520b3d88420e65e40100b60998503d856b5d29ec94f76e7de5aa2c64cc2176"}
Sep 08 15:32:57.186280 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.186253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"4e73f2224f5eb885e9542d56fb8d557b874f408b84ea030571d9d2b620e1d905"}
Sep 08 15:32:57.186341 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.186323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"d2559594703f28ac24de954956498725755724fbcff45a132e522f104d5db9b6"}
Sep 08 15:32:57.186388 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.186375    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"fc564d715ea195cdbd0c067c7c59ceebd257cacffd5ce5c9a17c69f6e2c6c614"}
Sep 08 15:32:57.186438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.186427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"da43fedb8cedb8536ac26f11c61e9ec8bfff28b2812961ca9ff3cd99749f3092"}
Sep 08 15:32:57.186479 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.186461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"b03838a6b6dcef07d76ae77208aa8ecbd0d472ab687dd593df0096f94e75c252"}
Sep 08 15:32:57.187698 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.187676    2563 generic.go:356] "Generic (PLEG): container finished" podID="847c4a7e-f4af-460b-9747-b082b1b323ec" containerID="97f7fe7367ea47777511b9bf90a7adfab0e89889a05e7f78ba78ffd0c245c3ef" exitCode=0
Sep 08 15:32:57.187785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.187712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerDied","Data":"97f7fe7367ea47777511b9bf90a7adfab0e89889a05e7f78ba78ffd0c245c3ef"}
Sep 08 15:32:57.209909 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.209865    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v2cz2" podStartSLOduration=2.958397104 podStartE2EDuration="22.20984845s" totalImagesPullingTime="19.251451346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:57.209709925 +0000 UTC m=+23.652902155" watchObservedRunningTime="2026-09-08 15:32:57.20984845 +0000 UTC m=+23.653040650"
Sep 08 15:32:57.264928 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.264871    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4xdfr" podStartSLOduration=2.240901842 podStartE2EDuration="22.264850521s" totalImagesPullingTime="20.023948679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:57.242400141 +0000 UTC m=+23.685592348" watchObservedRunningTime="2026-09-08 15:32:57.264850521 +0000 UTC m=+23.708042735"
Sep 08 15:32:57.293353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.293275    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qldnr" podStartSLOduration=2.295053917 podStartE2EDuration="22.293259441s" totalImagesPullingTime="19.998205524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:57.293003744 +0000 UTC m=+23.736195975" watchObservedRunningTime="2026-09-08 15:32:57.293259441 +0000 UTC m=+23.736451646"
Sep 08 15:32:57.293756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.293725    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qnrkp" podStartSLOduration=2.9886389810000002 podStartE2EDuration="22.293715593s" totalImagesPullingTime="19.305076612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:57.265729043 +0000 UTC m=+23.708921256" watchObservedRunningTime="2026-09-08 15:32:57.293715593 +0000 UTC m=+23.736907802"
Sep 08 15:32:57.955032 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.954898    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:57.955666 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.955632    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:57.983175 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:57.983108    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s9wdl" podStartSLOduration=2.928616648 podStartE2EDuration="22.983086691s" totalImagesPullingTime="20.054470043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:57.372268148 +0000 UTC m=+23.815460357" watchObservedRunningTime="2026-09-08 15:32:57.983086691 +0000 UTC m=+24.426278902"
Sep 08 15:32:58.094087 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.094045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:32:58.094276 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:58.094197    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:32:58.094276 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.094257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:32:58.094419 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:58.094380    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:32:58.149145 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.149110    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:58.150239 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.149935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4xdfr"
Sep 08 15:32:58.192621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.192560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mtxpj" event={"ID":"a13ff2a5-6eba-4fb7-b1a6-2e77810b94f1","Type":"ContainerStarted","Data":"5b2eab4d37a0f7c4a24c1a0d0c2cff3bda8a25bdb66f3ad4cbda78117bd09e1d"}
Sep 08 15:32:58.207785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.207735    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mtxpj" podStartSLOduration=3.173281376 podStartE2EDuration="23.207699002s" totalImagesPullingTime="20.034417626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:58.20720046 +0000 UTC m=+24.650392698" watchObservedRunningTime="2026-09-08 15:32:58.207699002 +0000 UTC m=+24.650891211"
Sep 08 15:32:58.260007 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:58.259982    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 15:32:59.076633 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.076531    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T15:32:58.260004935Z","UUID":"0394a024-c6e6-4aa9-9c90-39fd3150b43f","Handler":null,"Name":"","Endpoint":""}
Sep 08 15:32:59.077997 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.077978    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 15:32:59.078100 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.078009    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 15:32:59.094355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.094329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:32:59.094519 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:32:59.094463    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:32:59.197271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.197227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7" event={"ID":"da9eda6e-52f9-470c-a05a-e027f2cdc9ca","Type":"ContainerStarted","Data":"9d6a41fe5259e1f824a311c36c75e5b975d2a1469291a17012fc848b07893c5b"}
Sep 08 15:32:59.199622 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.199592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" event={"ID":"55bd377e98a5e435a35cefc9a800c096","Type":"ContainerStarted","Data":"ddf14013a03a4d9f83cff4d978dbfc56698c58cdedc3261392d8209254b1f02e"}
Sep 08 15:32:59.203705 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.203678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"f1541ad9e3703e81791a7123c7eb042cbf700c16182fa220bd9719bb3d6d8a4a"}
Sep 08 15:32:59.218592 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:32:59.218544    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-240.ec2.internal" podStartSLOduration=23.218527122 podStartE2EDuration="23.218527122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:32:59.218049312 +0000 UTC m=+25.661241523" watchObservedRunningTime="2026-09-08 15:32:59.218527122 +0000 UTC m=+25.661719333"
Sep 08 15:33:00.098118 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:00.098087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:00.098279 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:00.098202    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:33:00.098279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:00.098266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:00.098429 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:00.098409    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:33:00.208275 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:00.208240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7" event={"ID":"da9eda6e-52f9-470c-a05a-e027f2cdc9ca","Type":"ContainerStarted","Data":"41d576a6ae6636612bf7c101593f8ed8dc7110d563e269af21931fe1acbc0242"}
Sep 08 15:33:00.227454 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:00.227397    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4zs7" podStartSLOduration=2.525365116 podStartE2EDuration="25.227379985s" totalImagesPullingTime="22.702014869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:00.22696367 +0000 UTC m=+26.670155879" watchObservedRunningTime="2026-09-08 15:33:00.227379985 +0000 UTC m=+26.670572194"
Sep 08 15:33:01.094387 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:01.094351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:01.094563 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:01.094488    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:33:02.094263 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.094226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:02.094677 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.094237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:02.094677 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:02.094384    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:33:02.094677 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:02.094468    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:33:02.218469 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.218143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" event={"ID":"d91aa006-adab-4913-97fd-cbd01c28f594","Type":"ContainerStarted","Data":"f40435fa5f48d93790d2ee34a8faa4d9abe6ddc15ab523f9d9951e6402ceec13"}
Sep 08 15:33:02.218642 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.218464    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:33:02.218642 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.218552    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:33:02.218642 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.218598    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:33:02.236632 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.236612    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:33:02.239553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.239485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:33:02.247947 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:02.247887    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v" podStartSLOduration=7.143472667 podStartE2EDuration="27.247866506s" totalImagesPullingTime="20.104393839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:02.246783434 +0000 UTC m=+28.689975656" watchObservedRunningTime="2026-09-08 15:33:02.247866506 +0000 UTC m=+28.691058716"
Sep 08 15:33:03.094380 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:03.094346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:03.095121 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:03.094450    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:33:03.222536 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:03.222504    2563 generic.go:356] "Generic (PLEG): container finished" podID="847c4a7e-f4af-460b-9747-b082b1b323ec" containerID="83cbaa7fae3be6a8d64d3ba049338e2f71b6a53afb6f0501bfba0652d4712c82" exitCode=0
Sep 08 15:33:03.222687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:03.222583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerDied","Data":"83cbaa7fae3be6a8d64d3ba049338e2f71b6a53afb6f0501bfba0652d4712c82"}
Sep 08 15:33:04.061948 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:04.061586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dxkzl"]
Sep 08 15:33:04.062172 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:04.062147    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:04.062254 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:04.062188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pbvp6"]
Sep 08 15:33:04.062341 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:04.062269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:04.062341 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:04.062264    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:33:04.062454 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:04.062388    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:33:04.066338 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:04.065939    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gl426"]
Sep 08 15:33:04.066338 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:04.066060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:04.066338 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:04.066176    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:33:05.229339 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:05.229301    2563 generic.go:356] "Generic (PLEG): container finished" podID="847c4a7e-f4af-460b-9747-b082b1b323ec" containerID="14a824145b6ba147a4ce4a9ffe26b94476539d7b78552f356af561600b520877" exitCode=0
Sep 08 15:33:05.229339 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:05.229350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerDied","Data":"14a824145b6ba147a4ce4a9ffe26b94476539d7b78552f356af561600b520877"}
Sep 08 15:33:06.094410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:06.094335    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:06.094410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:06.094364    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:06.094410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:06.094332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:06.094617 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:06.094476    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:33:06.094617 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:06.094592    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:33:06.094682 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:06.094669    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:33:07.236929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:07.236832    2563 generic.go:356] "Generic (PLEG): container finished" podID="847c4a7e-f4af-460b-9747-b082b1b323ec" containerID="f812e69d14e4577235128041d198347e9680a2b39c21143db5abc736ee86a4be" exitCode=0
Sep 08 15:33:07.236929 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:07.236881    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerDied","Data":"f812e69d14e4577235128041d198347e9680a2b39c21143db5abc736ee86a4be"}
Sep 08 15:33:07.774955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:07.774747    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:07.775118 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:07.774979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:07.775118 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.774902    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 15:33:07.775118 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.775082    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:33:07.775118 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.775086    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret podName:13af96a0-8a7f-479c-887d-54f937793c41 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:39.775069222 +0000 UTC m=+66.218261422 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret") pod "global-pull-secret-syncer-pbvp6" (UID: "13af96a0-8a7f-479c-887d-54f937793c41") : object "kube-system"/"original-pull-secret" not registered
Sep 08 15:33:07.775267 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.775137    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:33:39.775120334 +0000 UTC m=+66.218312521 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 15:33:07.977828 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:07.977788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:07.978002 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.977971    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 15:33:07.978002 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.977992    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 15:33:07.978002 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.978003    2563 projected.go:196] Error preparing data for projected volume kube-api-access-slgk8 for pod openshift-network-diagnostics/network-check-target-gl426: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:33:07.978156 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:07.978145    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8 podName:15ee2f50-b774-4cd8-b706-0e6807cecbad nodeName:}" failed. No retries permitted until 2026-09-08 15:33:39.978127977 +0000 UTC m=+66.421320181 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-slgk8" (UniqueName: "kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8") pod "network-check-target-gl426" (UID: "15ee2f50-b774-4cd8-b706-0e6807cecbad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 15:33:08.095126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:08.095040    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:08.095313 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:08.095042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:08.095313 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:08.095169    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pbvp6" podUID="13af96a0-8a7f-479c-887d-54f937793c41"
Sep 08 15:33:08.095313 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:08.095181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:08.095313 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:08.095278    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dxkzl" podUID="3bdb3868-cedc-47d0-9c6c-cadb57a2b95e"
Sep 08 15:33:08.095519 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:08.095381    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gl426" podUID="15ee2f50-b774-4cd8-b706-0e6807cecbad"
Sep 08 15:33:09.325606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.325565    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-240.ec2.internal" event="NodeReady"
Sep 08 15:33:09.326047 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.325723    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 15:33:09.371343 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.371309    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zvvx9"]
Sep 08 15:33:09.391786 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.391097    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9bk86"]
Sep 08 15:33:09.391786 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.391196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.394678 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.394655    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 15:33:09.394823 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.394686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5prt2\""
Sep 08 15:33:09.395196 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.395171    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 15:33:09.409018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.408992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvvx9"]
Sep 08 15:33:09.409137 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.409036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9bk86"]
Sep 08 15:33:09.409137 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.409112    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.412501 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.412337    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 15:33:09.412501 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.412439    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nqnwb\""
Sep 08 15:33:09.412764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.412749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 15:33:09.414471 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.414447    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 15:33:09.489925 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.489887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706badb4-d406-4b28-8a58-8128f1a97764-tmp-dir\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.490113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.489973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cntxc\" (UniqueName: \"kubernetes.io/projected/706badb4-d406-4b28-8a58-8128f1a97764-kube-api-access-cntxc\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.490113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.490004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.490113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.490043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/706badb4-d406-4b28-8a58-8128f1a97764-config-volume\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.591317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706badb4-d406-4b28-8a58-8128f1a97764-tmp-dir\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.591487 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cntxc\" (UniqueName: \"kubernetes.io/projected/706badb4-d406-4b28-8a58-8128f1a97764-kube-api-access-cntxc\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.591487 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.591487 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65240601-6812-40d6-ba6f-1ef503d4f8ad-cert\") pod \"ingress-canary-9bk86\" (UID: \"65240601-6812-40d6-ba6f-1ef503d4f8ad\") " pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.591487 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8ppj\" (UniqueName: \"kubernetes.io/projected/65240601-6812-40d6-ba6f-1ef503d4f8ad-kube-api-access-j8ppj\") pod \"ingress-canary-9bk86\" (UID: \"65240601-6812-40d6-ba6f-1ef503d4f8ad\") " pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.591688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706badb4-d406-4b28-8a58-8128f1a97764-tmp-dir\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.591688 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:09.591548    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:09.591688 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:09.591610    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:10.091589209 +0000 UTC m=+36.534781409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:09.591688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.591613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/706badb4-d406-4b28-8a58-8128f1a97764-config-volume\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.592040 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.592020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/706badb4-d406-4b28-8a58-8128f1a97764-config-volume\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.602359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.602332    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cntxc\" (UniqueName: \"kubernetes.io/projected/706badb4-d406-4b28-8a58-8128f1a97764-kube-api-access-cntxc\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:09.692016 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.691980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65240601-6812-40d6-ba6f-1ef503d4f8ad-cert\") pod \"ingress-canary-9bk86\" (UID: \"65240601-6812-40d6-ba6f-1ef503d4f8ad\") " pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.692016 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.692031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8ppj\" (UniqueName: \"kubernetes.io/projected/65240601-6812-40d6-ba6f-1ef503d4f8ad-kube-api-access-j8ppj\") pod \"ingress-canary-9bk86\" (UID: \"65240601-6812-40d6-ba6f-1ef503d4f8ad\") " pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.694981 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.694955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65240601-6812-40d6-ba6f-1ef503d4f8ad-cert\") pod \"ingress-canary-9bk86\" (UID: \"65240601-6812-40d6-ba6f-1ef503d4f8ad\") " pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.701935 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.701902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8ppj\" (UniqueName: \"kubernetes.io/projected/65240601-6812-40d6-ba6f-1ef503d4f8ad-kube-api-access-j8ppj\") pod \"ingress-canary-9bk86\" (UID: \"65240601-6812-40d6-ba6f-1ef503d4f8ad\") " pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:09.718820 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:09.718785    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9bk86"
Sep 08 15:33:10.008644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.008569    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9bk86"]
Sep 08 15:33:10.013212 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:33:10.013180    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65240601_6812_40d6_ba6f_1ef503d4f8ad.slice/crio-02a32669b88cb3d9d652124f55020a07158e2984495073307826ff5c4b0e5611 WatchSource:0}: Error finding container 02a32669b88cb3d9d652124f55020a07158e2984495073307826ff5c4b0e5611: Status 404 returned error can't find the container with id 02a32669b88cb3d9d652124f55020a07158e2984495073307826ff5c4b0e5611
Sep 08 15:33:10.094307 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.094254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:10.094307 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.094303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:10.094523 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.094254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:10.095150 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.095123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:10.095297 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:10.095268    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:10.095377 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:10.095348    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:11.095326101 +0000 UTC m=+37.538518291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:10.097112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.097035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 15:33:10.097112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.097059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:33:10.097112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.097060    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xsqld\""
Sep 08 15:33:10.097807 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.097785    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xt4hq\""
Sep 08 15:33:10.097807 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.097780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 15:33:10.098255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.098237    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:33:10.246137 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:10.246100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9bk86" event={"ID":"65240601-6812-40d6-ba6f-1ef503d4f8ad","Type":"ContainerStarted","Data":"02a32669b88cb3d9d652124f55020a07158e2984495073307826ff5c4b0e5611"}
Sep 08 15:33:11.103863 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:11.103819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:11.104821 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:11.103985    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:11.104821 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:11.104051    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:13.104031223 +0000 UTC m=+39.547223413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:13.119407 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:13.119113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:13.119407 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:13.119270    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:13.119801 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:13.119456    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:17.119439631 +0000 UTC m=+43.562631817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:13.255863 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:13.255829    2563 generic.go:356] "Generic (PLEG): container finished" podID="847c4a7e-f4af-460b-9747-b082b1b323ec" containerID="f07ba699dd5d01c6ed2ae00b27708b796f21bf3a1f764adde2d2580d0e4ce937" exitCode=0
Sep 08 15:33:13.256014 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:13.255909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerDied","Data":"f07ba699dd5d01c6ed2ae00b27708b796f21bf3a1f764adde2d2580d0e4ce937"}
Sep 08 15:33:13.257352 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:13.257332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9bk86" event={"ID":"65240601-6812-40d6-ba6f-1ef503d4f8ad","Type":"ContainerStarted","Data":"2571bbc1a2317a750f4b647b50c896c983642f954586ef0927be767d605d937a"}
Sep 08 15:33:13.290115 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:13.290061    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9bk86" podStartSLOduration=1.516301215 podStartE2EDuration="4.290042694s" totalImagesPullingTime="2.773741479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:13.290015256 +0000 UTC m=+39.733207490" watchObservedRunningTime="2026-09-08 15:33:13.290042694 +0000 UTC m=+39.733234904"
Sep 08 15:33:14.263148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:14.263108    2563 generic.go:356] "Generic (PLEG): container finished" podID="847c4a7e-f4af-460b-9747-b082b1b323ec" containerID="b051f3eacb7f6c59e3518017abf0fc6d77793b85c34bfb4ac4e4de9d6ca07856" exitCode=0
Sep 08 15:33:14.263621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:14.263177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerDied","Data":"b051f3eacb7f6c59e3518017abf0fc6d77793b85c34bfb4ac4e4de9d6ca07856"}
Sep 08 15:33:15.268896 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:15.268859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" event={"ID":"847c4a7e-f4af-460b-9747-b082b1b323ec","Type":"ContainerStarted","Data":"37bb06f307f73cca2184285c775e95ab13f7ae18cda6cf440dca9ec66e101182"}
Sep 08 15:33:15.292188 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:15.292140    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mxdzc" podStartSLOduration=7.12618746 podStartE2EDuration="40.292122654s" totalImagesPullingTime="33.165935194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:15.291622266 +0000 UTC m=+41.734814475" watchObservedRunningTime="2026-09-08 15:33:15.292122654 +0000 UTC m=+41.735314880"
Sep 08 15:33:17.148442 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:17.148399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:17.148822 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:17.148534    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:17.148822 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:17.148594    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:25.148577986 +0000 UTC m=+51.591770174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:25.201794 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:25.201752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:25.202265 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:25.201879    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:25.202265 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:25.201928    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:41.201915191 +0000 UTC m=+67.645107391 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:34.242791 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:34.242760    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ktl7v"
Sep 08 15:33:39.799539 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:39.799494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:39.799930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:39.799551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:33:39.802099 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:39.802077    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 15:33:39.802174 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:39.802087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 15:33:39.810059 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:39.810036    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 15:33:39.810128 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:39.810117    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs podName:3bdb3868-cedc-47d0-9c6c-cadb57a2b95e nodeName:}" failed. No retries permitted until 2026-09-08 15:34:43.810093481 +0000 UTC m=+130.253285680 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs") pod "network-metrics-daemon-dxkzl" (UID: "3bdb3868-cedc-47d0-9c6c-cadb57a2b95e") : secret "metrics-daemon-secret" not found
Sep 08 15:33:39.813088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:39.813069    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/13af96a0-8a7f-479c-887d-54f937793c41-original-pull-secret\") pod \"global-pull-secret-syncer-pbvp6\" (UID: \"13af96a0-8a7f-479c-887d-54f937793c41\") " pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:39.816778 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:39.816759    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pbvp6"
Sep 08 15:33:40.000408 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.000348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:40.003078 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.003059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 15:33:40.012767 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.012743    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 15:33:40.023882 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.023858    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgk8\" (UniqueName: \"kubernetes.io/projected/15ee2f50-b774-4cd8-b706-0e6807cecbad-kube-api-access-slgk8\") pod \"network-check-target-gl426\" (UID: \"15ee2f50-b774-4cd8-b706-0e6807cecbad\") " pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:40.081203 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.081115    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pbvp6"]
Sep 08 15:33:40.084836 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:33:40.084814    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13af96a0_8a7f_479c_887d_54f937793c41.slice/crio-f22b0f732a4b80b5cd58016bb05c44ca8e6f9acd944304c653aa423aba776ea0 WatchSource:0}: Error finding container f22b0f732a4b80b5cd58016bb05c44ca8e6f9acd944304c653aa423aba776ea0: Status 404 returned error can't find the container with id f22b0f732a4b80b5cd58016bb05c44ca8e6f9acd944304c653aa423aba776ea0
Sep 08 15:33:40.124921 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.124890    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xt4hq\""
Sep 08 15:33:40.133090 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.133070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:40.333471 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.333379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pbvp6" event={"ID":"13af96a0-8a7f-479c-887d-54f937793c41","Type":"ContainerStarted","Data":"f22b0f732a4b80b5cd58016bb05c44ca8e6f9acd944304c653aa423aba776ea0"}
Sep 08 15:33:40.398691 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:40.398658    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gl426"]
Sep 08 15:33:40.401834 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:33:40.401808    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ee2f50_b774_4cd8_b706_0e6807cecbad.slice/crio-203a45f4af116290ef97b67ce036d22d0302ef22df310770a82edeee166396db WatchSource:0}: Error finding container 203a45f4af116290ef97b67ce036d22d0302ef22df310770a82edeee166396db: Status 404 returned error can't find the container with id 203a45f4af116290ef97b67ce036d22d0302ef22df310770a82edeee166396db
Sep 08 15:33:41.208743 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:41.208706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:33:41.209180 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:41.208870    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 15:33:41.209180 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:41.208946    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls podName:706badb4-d406-4b28-8a58-8128f1a97764 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:13.208925204 +0000 UTC m=+99.652117405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls") pod "dns-default-zvvx9" (UID: "706badb4-d406-4b28-8a58-8128f1a97764") : secret "dns-default-metrics-tls" not found
Sep 08 15:33:41.338540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:41.338490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gl426" event={"ID":"15ee2f50-b774-4cd8-b706-0e6807cecbad","Type":"ContainerStarted","Data":"203a45f4af116290ef97b67ce036d22d0302ef22df310770a82edeee166396db"}
Sep 08 15:33:44.366764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.366717    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pkwt5"]
Sep 08 15:33:44.369488 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.369460    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:44.371780 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.371755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 08 15:33:44.371780 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.371771    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 08 15:33:44.371983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.371809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mccpt\""
Sep 08 15:33:44.378816 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.378788    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pkwt5"]
Sep 08 15:33:44.439182 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.439142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:44.439426 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.439207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09eee509-c68d-4c0c-ab20-67f5589a9f23-nginx-conf\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:44.540499 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.540451    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:44.540693 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.540526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09eee509-c68d-4c0c-ab20-67f5589a9f23-nginx-conf\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:44.540693 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.540626    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:33:44.540801 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.540714    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert podName:09eee509-c68d-4c0c-ab20-67f5589a9f23 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:45.040691048 +0000 UTC m=+71.483883235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pkwt5" (UID: "09eee509-c68d-4c0c-ab20-67f5589a9f23") : secret "networking-console-plugin-cert" not found
Sep 08 15:33:44.541176 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.541147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09eee509-c68d-4c0c-ab20-67f5589a9f23-nginx-conf\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:44.580901 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.580860    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"]
Sep 08 15:33:44.582629 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.582608    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"]
Sep 08 15:33:44.582806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.582726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.584258 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.584241    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.585842 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.585803    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 15:33:44.585950 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.585846    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5w6h2\""
Sep 08 15:33:44.586157 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.586128    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 15:33:44.586271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.586171    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 15:33:44.586271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.586173    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 15:33:44.586430 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.586380    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 08 15:33:44.586430 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.586410    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:33:44.589348 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.589195    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 08 15:33:44.589348 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.589274    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-q656s\""
Sep 08 15:33:44.589506 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.589355    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 08 15:33:44.596043 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.596002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"]
Sep 08 15:33:44.596782 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.596764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"]
Sep 08 15:33:44.642203 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.642108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bad73a4b-a2fd-4520-ad6d-76467cc01927-config\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.642203 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.642170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mflxt\" (UniqueName: \"kubernetes.io/projected/bad73a4b-a2fd-4520-ad6d-76467cc01927-kube-api-access-mflxt\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.642455 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.642219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.642455 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.642432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bad73a4b-a2fd-4520-ad6d-76467cc01927-serving-cert\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.642565 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.642531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.642618 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.642600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5kkx\" (UniqueName: \"kubernetes.io/projected/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-kube-api-access-t5kkx\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.679752 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.679714    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77fb65d46d-j8tqb"]
Sep 08 15:33:44.681644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.681622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.685832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.685813    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 15:33:44.686698 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.686464    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 15:33:44.686698 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.686568    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lpbdz\""
Sep 08 15:33:44.686928 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.686911    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 15:33:44.686997 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.686974    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 15:33:44.687053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.686985    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 15:33:44.687665 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.687649    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 15:33:44.696902 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.696878    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77fb65d46d-j8tqb"]
Sep 08 15:33:44.743572 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5kkx\" (UniqueName: \"kubernetes.io/projected/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-kube-api-access-t5kkx\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.743770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.743770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pm54\" (UniqueName: \"kubernetes.io/projected/66e63dad-53d0-4dc5-bb24-421801c5b627-kube-api-access-9pm54\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.743770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bad73a4b-a2fd-4520-ad6d-76467cc01927-config\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.743770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mflxt\" (UniqueName: \"kubernetes.io/projected/bad73a4b-a2fd-4520-ad6d-76467cc01927-kube-api-access-mflxt\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.743770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-default-certificate\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bad73a4b-a2fd-4520-ad6d-76467cc01927-serving-cert\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.743900    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.743945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-stats-auth\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.744025 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.743984    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls podName:d99c67bb-3c5e-41fb-84af-6d4441c8ea78 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:45.24395858 +0000 UTC m=+71.687150785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zxhhj" (UID: "d99c67bb-3c5e-41fb-84af-6d4441c8ea78") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:44.744461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.744316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bad73a4b-a2fd-4520-ad6d-76467cc01927-config\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.744801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.744775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.746488 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.746462    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bad73a4b-a2fd-4520-ad6d-76467cc01927-serving-cert\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.755597 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.755574    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5kkx\" (UniqueName: \"kubernetes.io/projected/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-kube-api-access-t5kkx\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:44.755799 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.755784    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mflxt\" (UniqueName: \"kubernetes.io/projected/bad73a4b-a2fd-4520-ad6d-76467cc01927-kube-api-access-mflxt\") pod \"service-ca-operator-859f94c4b-tbbgh\" (UID: \"bad73a4b-a2fd-4520-ad6d-76467cc01927\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:44.844736 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.844678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.844917 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.844878    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:33:44.844961 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.844881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pm54\" (UniqueName: \"kubernetes.io/projected/66e63dad-53d0-4dc5-bb24-421801c5b627-kube-api-access-9pm54\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.844961 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.844955    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:45.344933665 +0000 UTC m=+71.788125868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : secret "router-metrics-certs-default" not found
Sep 08 15:33:44.845037 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.845027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-default-certificate\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.845071 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.845057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.845196 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:44.845177    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:45.345164484 +0000 UTC m=+71.788356672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : configmap references non-existent config key: service-ca.crt
Sep 08 15:33:44.845254 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.845238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-stats-auth\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.848160 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.848119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-stats-auth\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.848160 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.848130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-default-certificate\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.854840 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.854816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pm54\" (UniqueName: \"kubernetes.io/projected/66e63dad-53d0-4dc5-bb24-421801c5b627-kube-api-access-9pm54\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:44.901663 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:44.901566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"
Sep 08 15:33:45.047612 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.047577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:45.047768 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.047692    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:33:45.047816 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.047771    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert podName:09eee509-c68d-4c0c-ab20-67f5589a9f23 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:46.047748837 +0000 UTC m=+72.490941042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pkwt5" (UID: "09eee509-c68d-4c0c-ab20-67f5589a9f23") : secret "networking-console-plugin-cert" not found
Sep 08 15:33:45.250249 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.250204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:45.250466 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.250332    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:45.250466 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.250389    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls podName:d99c67bb-3c5e-41fb-84af-6d4441c8ea78 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:46.250374415 +0000 UTC m=+72.693566615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zxhhj" (UID: "d99c67bb-3c5e-41fb-84af-6d4441c8ea78") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:45.342455 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.342410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh"]
Sep 08 15:33:45.347222 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:33:45.347193    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad73a4b_a2fd_4520_ad6d_76467cc01927.slice/crio-2d05766c7436904195dcd3ca21d620f6b807269d81aeacb862f449bc455a9bab WatchSource:0}: Error finding container 2d05766c7436904195dcd3ca21d620f6b807269d81aeacb862f449bc455a9bab: Status 404 returned error can't find the container with id 2d05766c7436904195dcd3ca21d620f6b807269d81aeacb862f449bc455a9bab
Sep 08 15:33:45.350199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.350168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pbvp6" event={"ID":"13af96a0-8a7f-479c-887d-54f937793c41","Type":"ContainerStarted","Data":"ed94c6c21aff82a454ac233fd8e6dda2fd31e17188df89b7fc58518bf53ad3a0"}
Sep 08 15:33:45.350587 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.350565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:45.350671 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.350643    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:33:45.350671 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.350661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:45.350782 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.350699    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:46.350680053 +0000 UTC m=+72.793872241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : secret "router-metrics-certs-default" not found
Sep 08 15:33:45.350782 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:45.350774    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:46.35075682 +0000 UTC m=+72.793949009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : configmap references non-existent config key: service-ca.crt
Sep 08 15:33:45.351824 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.351794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gl426" event={"ID":"15ee2f50-b774-4cd8-b706-0e6807cecbad","Type":"ContainerStarted","Data":"325ec48c3c6e68e40c8d6841c91f72ead38d37521573c7e3b71497e7a30f8cdb"}
Sep 08 15:33:45.351948 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.351933    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:33:45.364707 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.364654    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pbvp6" podStartSLOduration=65.365669016 podStartE2EDuration="1m10.364638339s" totalImagesPullingTime="4.998969323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:45.36404738 +0000 UTC m=+71.807239610" watchObservedRunningTime="2026-09-08 15:33:45.364638339 +0000 UTC m=+71.807830548"
Sep 08 15:33:45.376813 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:45.376763    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gl426" podStartSLOduration=65.693675644 podStartE2EDuration="1m10.376748825s" totalImagesPullingTime="4.683073181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:45.37644085 +0000 UTC m=+71.819633081" watchObservedRunningTime="2026-09-08 15:33:45.376748825 +0000 UTC m=+71.819941032"
Sep 08 15:33:46.057086 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:46.057045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:46.057274 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.057160    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:33:46.057274 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.057215    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert podName:09eee509-c68d-4c0c-ab20-67f5589a9f23 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:48.057201089 +0000 UTC m=+74.500393276 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pkwt5" (UID: "09eee509-c68d-4c0c-ab20-67f5589a9f23") : secret "networking-console-plugin-cert" not found
Sep 08 15:33:46.258632 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:46.258586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:46.258808 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.258714    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:46.258808 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.258773    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls podName:d99c67bb-3c5e-41fb-84af-6d4441c8ea78 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:48.258759904 +0000 UTC m=+74.701952090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zxhhj" (UID: "d99c67bb-3c5e-41fb-84af-6d4441c8ea78") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:46.355724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:46.355633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh" event={"ID":"bad73a4b-a2fd-4520-ad6d-76467cc01927","Type":"ContainerStarted","Data":"2d05766c7436904195dcd3ca21d620f6b807269d81aeacb862f449bc455a9bab"}
Sep 08 15:33:46.359346 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:46.359321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:46.359503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:46.359403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:46.359503 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.359474    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:33:46.359597 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.359527    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:48.359509338 +0000 UTC m=+74.802701528 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : configmap references non-existent config key: service-ca.crt
Sep 08 15:33:46.359597 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:46.359548    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:48.359538068 +0000 UTC m=+74.802730261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : secret "router-metrics-certs-default" not found
Sep 08 15:33:48.074889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.074796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:48.075276 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.074944    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:33:48.075276 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.075012    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert podName:09eee509-c68d-4c0c-ab20-67f5589a9f23 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:52.074994189 +0000 UTC m=+78.518186398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pkwt5" (UID: "09eee509-c68d-4c0c-ab20-67f5589a9f23") : secret "networking-console-plugin-cert" not found
Sep 08 15:33:48.084955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.084912    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-shh8w"]
Sep 08 15:33:48.108177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.108142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.110845 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.110820    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l67b9\""
Sep 08 15:33:48.110986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.110848    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 15:33:48.110986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.110872    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 15:33:48.111537 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.111513    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 15:33:48.111696 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.111519    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 15:33:48.112250 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.112230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shh8w"]
Sep 08 15:33:48.175595 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.175558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qnd2\" (UniqueName: \"kubernetes.io/projected/135ab67e-b78a-4307-95db-4dea8480e412-kube-api-access-4qnd2\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.175595 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.175602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.175867 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.175638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.175867 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.175736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/135ab67e-b78a-4307-95db-4dea8480e412-crio-socket\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.175867 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.175775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/135ab67e-b78a-4307-95db-4dea8480e412-data-volume\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277217 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277421 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277421 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.277332    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:33:48.277421 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.277378    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:48.277421 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/135ab67e-b78a-4307-95db-4dea8480e412-crio-socket\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277421 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.277420    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:48.777398535 +0000 UTC m=+75.220590736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : configmap "kube-rbac-proxy" not found
Sep 08 15:33:48.277674 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/135ab67e-b78a-4307-95db-4dea8480e412-crio-socket\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277674 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.277479    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:48.777457935 +0000 UTC m=+75.220650138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:48.277674 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/135ab67e-b78a-4307-95db-4dea8480e412-data-volume\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277674 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:48.277674 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.277617    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:48.277864 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.277677    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls podName:d99c67bb-3c5e-41fb-84af-6d4441c8ea78 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:52.277665099 +0000 UTC m=+78.720857294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zxhhj" (UID: "d99c67bb-3c5e-41fb-84af-6d4441c8ea78") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:48.277864 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277820    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qnd2\" (UniqueName: \"kubernetes.io/projected/135ab67e-b78a-4307-95db-4dea8480e412-kube-api-access-4qnd2\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.277949 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.277933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/135ab67e-b78a-4307-95db-4dea8480e412-data-volume\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.288682 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.288648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qnd2\" (UniqueName: \"kubernetes.io/projected/135ab67e-b78a-4307-95db-4dea8480e412-kube-api-access-4qnd2\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.362651 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.362559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh" event={"ID":"bad73a4b-a2fd-4520-ad6d-76467cc01927","Type":"ContainerStarted","Data":"83769695a2819ef189e8f262d8c7ce0713fe73a32858a8f9743acd7c8f753512"}
Sep 08 15:33:48.378246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.378211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:48.378422 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.378353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:48.378422 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.378379    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:33:48.378539 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.378461    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:52.378441936 +0000 UTC m=+78.821634141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : secret "router-metrics-certs-default" not found
Sep 08 15:33:48.378539 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.378510    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:52.378491948 +0000 UTC m=+78.821684150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : configmap references non-existent config key: service-ca.crt
Sep 08 15:33:48.379063 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.379012    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh" podStartSLOduration=1.9209029 podStartE2EDuration="4.378995627s" totalImagesPullingTime="2.458092727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:48.378045652 +0000 UTC m=+74.821237864" watchObservedRunningTime="2026-09-08 15:33:48.378995627 +0000 UTC m=+74.822187839"
Sep 08 15:33:48.782516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.782467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.782721 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:48.782542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:48.782721 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.782592    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:33:48.782721 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.782658    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:49.782641388 +0000 UTC m=+76.225833575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : configmap "kube-rbac-proxy" not found
Sep 08 15:33:48.782721 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.782696    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:48.782891 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:48.782757    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:49.782740585 +0000 UTC m=+76.225932787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:49.790175 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:49.790139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:49.790665 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:49.790244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:49.790665 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:49.790337    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:33:49.790665 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:49.790356    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:49.790665 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:49.790392    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:51.790377725 +0000 UTC m=+78.233569911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : configmap "kube-rbac-proxy" not found
Sep 08 15:33:49.790665 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:49.790429    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:51.790409709 +0000 UTC m=+78.233601915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:50.035508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:50.035477    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qldnr_a82652f7-3dc5-4132-9dd6-02e12e6a95cd/dns-node-resolver/0.log"
Sep 08 15:33:51.231088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.231058    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v2cz2_3f439e87-2fcb-430d-960b-eb74a8e5efdd/node-ca/0.log"
Sep 08 15:33:51.617802 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.617727    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xm6x8"]
Sep 08 15:33:51.619856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.619836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.621998 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.621977    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 08 15:33:51.622841 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.622815    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 08 15:33:51.623011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.622991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-x5w2m\""
Sep 08 15:33:51.623098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.623066    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 08 15:33:51.623098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.622991    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 08 15:33:51.633118 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.633097    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xm6x8"]
Sep 08 15:33:51.636460 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.636438    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9bk86_65240601-6812-40d6-ba6f-1ef503d4f8ad/serve-healthcheck-canary/0.log"
Sep 08 15:33:51.705000 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.704955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qcmz\" (UniqueName: \"kubernetes.io/projected/005ae319-0dc0-45a8-9396-04714c4055f2-kube-api-access-9qcmz\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.705000 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.705002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/005ae319-0dc0-45a8-9396-04714c4055f2-signing-cabundle\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.705212 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.705147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/005ae319-0dc0-45a8-9396-04714c4055f2-signing-key\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.806456 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.806409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/005ae319-0dc0-45a8-9396-04714c4055f2-signing-key\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.806456 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.806459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:51.806702 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.806489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qcmz\" (UniqueName: \"kubernetes.io/projected/005ae319-0dc0-45a8-9396-04714c4055f2-kube-api-access-9qcmz\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.806702 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.806509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:51.806702 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.806531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/005ae319-0dc0-45a8-9396-04714c4055f2-signing-cabundle\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.806702 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:51.806625    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:51.806702 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:51.806703    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:55.806682052 +0000 UTC m=+82.249874253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:51.806953 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:51.806626    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:33:51.806953 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:51.806776    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:33:55.806759901 +0000 UTC m=+82.249952097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : configmap "kube-rbac-proxy" not found
Sep 08 15:33:51.807230 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.807210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/005ae319-0dc0-45a8-9396-04714c4055f2-signing-cabundle\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.808824 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.808803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/005ae319-0dc0-45a8-9396-04714c4055f2-signing-key\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.815547 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.815523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qcmz\" (UniqueName: \"kubernetes.io/projected/005ae319-0dc0-45a8-9396-04714c4055f2-kube-api-access-9qcmz\") pod \"service-ca-75d99c484-xm6x8\" (UID: \"005ae319-0dc0-45a8-9396-04714c4055f2\") " pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:51.928996 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:51.928963    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-xm6x8"
Sep 08 15:33:52.108775 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.108741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:33:52.108972 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.108900    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:33:52.109100 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.108979    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert podName:09eee509-c68d-4c0c-ab20-67f5589a9f23 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:00.108958364 +0000 UTC m=+86.552150557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pkwt5" (UID: "09eee509-c68d-4c0c-ab20-67f5589a9f23") : secret "networking-console-plugin-cert" not found
Sep 08 15:33:52.207324 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.207214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-xm6x8"]
Sep 08 15:33:52.214356 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:33:52.214324    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005ae319_0dc0_45a8_9396_04714c4055f2.slice/crio-6f2d3fe8f27e160ccdee38c8883cba17b9fb9257f7c525971e89c34b298e48a6 WatchSource:0}: Error finding container 6f2d3fe8f27e160ccdee38c8883cba17b9fb9257f7c525971e89c34b298e48a6: Status 404 returned error can't find the container with id 6f2d3fe8f27e160ccdee38c8883cba17b9fb9257f7c525971e89c34b298e48a6
Sep 08 15:33:52.311794 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.311757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:33:52.312206 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.311935    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:52.312206 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.312020    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls podName:d99c67bb-3c5e-41fb-84af-6d4441c8ea78 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:00.311997253 +0000 UTC m=+86.755189459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zxhhj" (UID: "d99c67bb-3c5e-41fb-84af-6d4441c8ea78") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:33:52.374569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.374535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-xm6x8" event={"ID":"005ae319-0dc0-45a8-9396-04714c4055f2","Type":"ContainerStarted","Data":"b711ba2b8e140c1a4056603a9f2b980aea6fd069bcba84c364bcfcd7d49bdd20"}
Sep 08 15:33:52.374729 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.374588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-xm6x8" event={"ID":"005ae319-0dc0-45a8-9396-04714c4055f2","Type":"ContainerStarted","Data":"6f2d3fe8f27e160ccdee38c8883cba17b9fb9257f7c525971e89c34b298e48a6"}
Sep 08 15:33:52.390502 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.390451    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-xm6x8" podStartSLOduration=1.390434089 podStartE2EDuration="1.390434089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:33:52.390406935 +0000 UTC m=+78.833599172" watchObservedRunningTime="2026-09-08 15:33:52.390434089 +0000 UTC m=+78.833626298"
Sep 08 15:33:52.413051 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.413014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:52.413227 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:52.413111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:33:52.413227 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.413168    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 15:33:52.413332 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.413235    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:00.413220123 +0000 UTC m=+86.856412310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : secret "router-metrics-certs-default" not found
Sep 08 15:33:52.413332 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:52.413249    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle podName:66e63dad-53d0-4dc5-bb24-421801c5b627 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:00.413242578 +0000 UTC m=+86.856434764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle") pod "router-default-77fb65d46d-j8tqb" (UID: "66e63dad-53d0-4dc5-bb24-421801c5b627") : configmap references non-existent config key: service-ca.crt
Sep 08 15:33:55.842907 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:55.842752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:55.842907 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:33:55.842811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:33:55.842907 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:55.842882    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 15:33:55.842907 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:55.842905    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:55.843541 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:55.842961    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:03.842944769 +0000 UTC m=+90.286136974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : secret "insights-runtime-extractor-tls" not found
Sep 08 15:33:55.843541 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:33:55.842980    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm podName:135ab67e-b78a-4307-95db-4dea8480e412 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:03.842969624 +0000 UTC m=+90.286161814 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shh8w" (UID: "135ab67e-b78a-4307-95db-4dea8480e412") : configmap "kube-rbac-proxy" not found
Sep 08 15:34:00.184167 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.184117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:34:00.184674 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:00.184238    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 15:34:00.184674 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:00.184316    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert podName:09eee509-c68d-4c0c-ab20-67f5589a9f23 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:16.184278322 +0000 UTC m=+102.627470509 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-pkwt5" (UID: "09eee509-c68d-4c0c-ab20-67f5589a9f23") : secret "networking-console-plugin-cert" not found
Sep 08 15:34:00.385136 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.385091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:34:00.385330 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:00.385246    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 15:34:00.385384 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:00.385339    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls podName:d99c67bb-3c5e-41fb-84af-6d4441c8ea78 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:16.385321025 +0000 UTC m=+102.828513213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-zxhhj" (UID: "d99c67bb-3c5e-41fb-84af-6d4441c8ea78") : secret "cluster-monitoring-operator-tls" not found
Sep 08 15:34:00.485835 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.485732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:00.486006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.485846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:00.486558 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.486525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e63dad-53d0-4dc5-bb24-421801c5b627-service-ca-bundle\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:00.488124 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.488105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66e63dad-53d0-4dc5-bb24-421801c5b627-metrics-certs\") pod \"router-default-77fb65d46d-j8tqb\" (UID: \"66e63dad-53d0-4dc5-bb24-421801c5b627\") " pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:00.592801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.592756    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:00.868529 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:00.868454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77fb65d46d-j8tqb"]
Sep 08 15:34:00.871647 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:00.871622    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e63dad_53d0_4dc5_bb24_421801c5b627.slice/crio-cb45479c07e7e1be0e9cf6cad79ffbe014be0f414676a2a773d719833ac09cf9 WatchSource:0}: Error finding container cb45479c07e7e1be0e9cf6cad79ffbe014be0f414676a2a773d719833ac09cf9: Status 404 returned error can't find the container with id cb45479c07e7e1be0e9cf6cad79ffbe014be0f414676a2a773d719833ac09cf9
Sep 08 15:34:01.399964 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:01.399927    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77fb65d46d-j8tqb" event={"ID":"66e63dad-53d0-4dc5-bb24-421801c5b627","Type":"ContainerStarted","Data":"e1704750493be36b8e031831700788221e42de7b19e6d02773346ff527275c8f"}
Sep 08 15:34:01.400378 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:01.399976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77fb65d46d-j8tqb" event={"ID":"66e63dad-53d0-4dc5-bb24-421801c5b627","Type":"ContainerStarted","Data":"cb45479c07e7e1be0e9cf6cad79ffbe014be0f414676a2a773d719833ac09cf9"}
Sep 08 15:34:01.438917 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:01.438858    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77fb65d46d-j8tqb" podStartSLOduration=17.438839893 podStartE2EDuration="17.438839893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:01.438277335 +0000 UTC m=+87.881469578" watchObservedRunningTime="2026-09-08 15:34:01.438839893 +0000 UTC m=+87.882032102"
Sep 08 15:34:01.593777 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:01.593731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:01.596540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:01.596515    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:02.404301 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:02.404270    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:02.405555 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:02.405532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77fb65d46d-j8tqb"
Sep 08 15:34:03.914053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:03.914006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:34:03.914053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:03.914057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:34:03.914580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:03.914560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/135ab67e-b78a-4307-95db-4dea8480e412-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:34:03.916414 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:03.916393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/135ab67e-b78a-4307-95db-4dea8480e412-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shh8w\" (UID: \"135ab67e-b78a-4307-95db-4dea8480e412\") " pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:34:04.018214 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:04.018184    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shh8w"
Sep 08 15:34:04.288921 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:04.288887    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shh8w"]
Sep 08 15:34:04.292433 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:04.292400    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135ab67e_b78a_4307_95db_4dea8480e412.slice/crio-fd8675c1bb69f103aa3c08f13fa5b06d68d8552e1e6ccce67947525e7ba9a80e WatchSource:0}: Error finding container fd8675c1bb69f103aa3c08f13fa5b06d68d8552e1e6ccce67947525e7ba9a80e: Status 404 returned error can't find the container with id fd8675c1bb69f103aa3c08f13fa5b06d68d8552e1e6ccce67947525e7ba9a80e
Sep 08 15:34:04.411822 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:04.411786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shh8w" event={"ID":"135ab67e-b78a-4307-95db-4dea8480e412","Type":"ContainerStarted","Data":"673e3f40770b8827d774c91ac1061a40e35cfd051e4e1ef7248330ea54f2bf68"}
Sep 08 15:34:04.412022 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:04.411857    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shh8w" event={"ID":"135ab67e-b78a-4307-95db-4dea8480e412","Type":"ContainerStarted","Data":"fd8675c1bb69f103aa3c08f13fa5b06d68d8552e1e6ccce67947525e7ba9a80e"}
Sep 08 15:34:05.419009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:05.418916    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shh8w" event={"ID":"135ab67e-b78a-4307-95db-4dea8480e412","Type":"ContainerStarted","Data":"d43de3883f23045093d7036cefc87f111949b70e405f2e87e48826e1cf763b35"}
Sep 08 15:34:07.427526 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:07.427491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shh8w" event={"ID":"135ab67e-b78a-4307-95db-4dea8480e412","Type":"ContainerStarted","Data":"dbc790a23fb608f76925023fe11319029ef39bb0029f737eb7f0e81a16a25928"}
Sep 08 15:34:07.445370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:07.445312    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-shh8w" podStartSLOduration=16.774162308 podStartE2EDuration="19.445273658s" totalImagesPullingTime="2.67111135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 15:33:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:07.444931377 +0000 UTC m=+93.888123630" watchObservedRunningTime="2026-09-08 15:34:07.445273658 +0000 UTC m=+93.888465860"
Sep 08 15:34:10.867442 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.867402    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"]
Sep 08 15:34:10.869589 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.869570    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:10.872140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.872114    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8nlfz\""
Sep 08 15:34:10.872255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.872136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 15:34:10.872663 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.872647    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 15:34:10.872741 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.872653    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 15:34:10.873259 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.873240    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 08 15:34:10.878578 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.878554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"]
Sep 08 15:34:10.968403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.968363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc926158-bf3c-4d89-8033-8dbf70bdc702-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s\" (UID: \"bc926158-bf3c-4d89-8033-8dbf70bdc702\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:10.968572 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:10.968436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkwg6\" (UniqueName: \"kubernetes.io/projected/bc926158-bf3c-4d89-8033-8dbf70bdc702-kube-api-access-fkwg6\") pod \"managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s\" (UID: \"bc926158-bf3c-4d89-8033-8dbf70bdc702\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:11.069343 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:11.069306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwg6\" (UniqueName: \"kubernetes.io/projected/bc926158-bf3c-4d89-8033-8dbf70bdc702-kube-api-access-fkwg6\") pod \"managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s\" (UID: \"bc926158-bf3c-4d89-8033-8dbf70bdc702\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:11.069565 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:11.069381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc926158-bf3c-4d89-8033-8dbf70bdc702-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s\" (UID: \"bc926158-bf3c-4d89-8033-8dbf70bdc702\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:11.071874 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:11.071847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bc926158-bf3c-4d89-8033-8dbf70bdc702-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s\" (UID: \"bc926158-bf3c-4d89-8033-8dbf70bdc702\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:11.081926 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:11.081903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkwg6\" (UniqueName: \"kubernetes.io/projected/bc926158-bf3c-4d89-8033-8dbf70bdc702-kube-api-access-fkwg6\") pod \"managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s\" (UID: \"bc926158-bf3c-4d89-8033-8dbf70bdc702\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:11.188190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:11.188160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"
Sep 08 15:34:11.459832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:11.459760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s"]
Sep 08 15:34:11.462841 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:11.462813    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc926158_bf3c_4d89_8033_8dbf70bdc702.slice/crio-812151263a5e743efc7eda79841baf8cef7e3e87915f765c775fbbac8ea60461 WatchSource:0}: Error finding container 812151263a5e743efc7eda79841baf8cef7e3e87915f765c775fbbac8ea60461: Status 404 returned error can't find the container with id 812151263a5e743efc7eda79841baf8cef7e3e87915f765c775fbbac8ea60461
Sep 08 15:34:12.446134 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:12.446073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s" event={"ID":"bc926158-bf3c-4d89-8033-8dbf70bdc702","Type":"ContainerStarted","Data":"812151263a5e743efc7eda79841baf8cef7e3e87915f765c775fbbac8ea60461"}
Sep 08 15:34:13.288743 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:13.288699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:34:13.291375 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:13.291345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/706badb4-d406-4b28-8a58-8128f1a97764-metrics-tls\") pod \"dns-default-zvvx9\" (UID: \"706badb4-d406-4b28-8a58-8128f1a97764\") " pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:34:13.307387 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:13.307351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5prt2\""
Sep 08 15:34:13.316109 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:13.316081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:34:14.124694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:14.124603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvvx9"]
Sep 08 15:34:14.128696 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:14.128665    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706badb4_d406_4b28_8a58_8128f1a97764.slice/crio-5c6494482af3614cb7cb2c3a276c8dc8d37d8e7803a18504b7697f635c4524e0 WatchSource:0}: Error finding container 5c6494482af3614cb7cb2c3a276c8dc8d37d8e7803a18504b7697f635c4524e0: Status 404 returned error can't find the container with id 5c6494482af3614cb7cb2c3a276c8dc8d37d8e7803a18504b7697f635c4524e0
Sep 08 15:34:14.455432 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:14.455389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s" event={"ID":"bc926158-bf3c-4d89-8033-8dbf70bdc702","Type":"ContainerStarted","Data":"93df5403ccef51a91ae8b96c5c15c69728865e71df1fb8801f13ff58e6f04dc8"}
Sep 08 15:34:14.456614 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:14.456590    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvvx9" event={"ID":"706badb4-d406-4b28-8a58-8128f1a97764","Type":"ContainerStarted","Data":"5c6494482af3614cb7cb2c3a276c8dc8d37d8e7803a18504b7697f635c4524e0"}
Sep 08 15:34:14.471718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:14.471663    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-64fc9cbb9b-tb68s" podStartSLOduration=2.046219806 podStartE2EDuration="4.471643885s" totalImagesPullingTime="2.425424079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:14.470981741 +0000 UTC m=+100.914173951" watchObservedRunningTime="2026-09-08 15:34:14.471643885 +0000 UTC m=+100.914836094"
Sep 08 15:34:16.215801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.215763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:34:16.218126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.218103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09eee509-c68d-4c0c-ab20-67f5589a9f23-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-pkwt5\" (UID: \"09eee509-c68d-4c0c-ab20-67f5589a9f23\") " pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:34:16.359021 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.358995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gl426"
Sep 08 15:34:16.417611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.417578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:34:16.419942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.419918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d99c67bb-3c5e-41fb-84af-6d4441c8ea78-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-zxhhj\" (UID: \"d99c67bb-3c5e-41fb-84af-6d4441c8ea78\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:34:16.465435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.465403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvvx9" event={"ID":"706badb4-d406-4b28-8a58-8128f1a97764","Type":"ContainerStarted","Data":"192a81fc034526ece5526b24b0f413ef929b75752d6fc69ec78528c8799b0a3c"}
Sep 08 15:34:16.465638 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.465454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvvx9" event={"ID":"706badb4-d406-4b28-8a58-8128f1a97764","Type":"ContainerStarted","Data":"56d5ccffe53ea3537b588fe6dcdc07ecccb475837d4813edda97598c49c8d223"}
Sep 08 15:34:16.465638 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.465542    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:34:16.479591 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.479565    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5"
Sep 08 15:34:16.483117 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.483074    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zvvx9" podStartSLOduration=65.537323481 podStartE2EDuration="1m7.48305887s" totalImagesPullingTime="1.945735389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:16.482281277 +0000 UTC m=+102.925473530" watchObservedRunningTime="2026-09-08 15:34:16.48305887 +0000 UTC m=+102.926251079"
Sep 08 15:34:16.694653 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.694624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"
Sep 08 15:34:16.767506 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.767468    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-pkwt5"]
Sep 08 15:34:16.770785 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:16.770757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09eee509_c68d_4c0c_ab20_67f5589a9f23.slice/crio-55325325ff220d9ff5f25a29f7b882a1c1384533fcfc70fe86fb93392b8e33c2 WatchSource:0}: Error finding container 55325325ff220d9ff5f25a29f7b882a1c1384533fcfc70fe86fb93392b8e33c2: Status 404 returned error can't find the container with id 55325325ff220d9ff5f25a29f7b882a1c1384533fcfc70fe86fb93392b8e33c2
Sep 08 15:34:16.997883 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:16.997799    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj"]
Sep 08 15:34:17.000779 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:17.000752    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99c67bb_3c5e_41fb_84af_6d4441c8ea78.slice/crio-28386f85ce4a6f6e7a2be31b3162f102eff4b49618c2fe94a0fb8b4ef3bd668a WatchSource:0}: Error finding container 28386f85ce4a6f6e7a2be31b3162f102eff4b49618c2fe94a0fb8b4ef3bd668a: Status 404 returned error can't find the container with id 28386f85ce4a6f6e7a2be31b3162f102eff4b49618c2fe94a0fb8b4ef3bd668a
Sep 08 15:34:17.469636 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:17.469600    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5" event={"ID":"09eee509-c68d-4c0c-ab20-67f5589a9f23","Type":"ContainerStarted","Data":"55325325ff220d9ff5f25a29f7b882a1c1384533fcfc70fe86fb93392b8e33c2"}
Sep 08 15:34:17.470814 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:17.470789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj" event={"ID":"d99c67bb-3c5e-41fb-84af-6d4441c8ea78","Type":"ContainerStarted","Data":"28386f85ce4a6f6e7a2be31b3162f102eff4b49618c2fe94a0fb8b4ef3bd668a"}
Sep 08 15:34:19.930066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:19.930029    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"]
Sep 08 15:34:19.931937 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:19.931922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:19.935164 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:19.935141    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 08 15:34:19.935311 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:19.935178    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7lnl7\""
Sep 08 15:34:19.941837 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:19.941813    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"]
Sep 08 15:34:20.047447 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.047401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bcdeb6e7-d979-4470-90e5-62fe8324dd90-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2hg4r\" (UID: \"bcdeb6e7-d979-4470-90e5-62fe8324dd90\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:20.148621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.148577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bcdeb6e7-d979-4470-90e5-62fe8324dd90-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2hg4r\" (UID: \"bcdeb6e7-d979-4470-90e5-62fe8324dd90\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:20.148831 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:20.148732    2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 08 15:34:20.148831 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:20.148806    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcdeb6e7-d979-4470-90e5-62fe8324dd90-tls-certificates podName:bcdeb6e7-d979-4470-90e5-62fe8324dd90 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:20.648788399 +0000 UTC m=+107.091980586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/bcdeb6e7-d979-4470-90e5-62fe8324dd90-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-2hg4r" (UID: "bcdeb6e7-d979-4470-90e5-62fe8324dd90") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 08 15:34:20.480900 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.480853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj" event={"ID":"d99c67bb-3c5e-41fb-84af-6d4441c8ea78","Type":"ContainerStarted","Data":"c2b82731d21cb6c4e02da1a2d3f7ff832a83275d794b7082e9b262635e80dda5"}
Sep 08 15:34:20.482390 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.482361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5" event={"ID":"09eee509-c68d-4c0c-ab20-67f5589a9f23","Type":"ContainerStarted","Data":"f15aed968912f7bb64455f9dd7bb8057533d49470f315ccc0463effc97fd753a"}
Sep 08 15:34:20.497192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.497144    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-zxhhj" podStartSLOduration=34.062861092 podStartE2EDuration="36.49713129s" totalImagesPullingTime="2.434270198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:20.496776451 +0000 UTC m=+106.939968693" watchObservedRunningTime="2026-09-08 15:34:20.49713129 +0000 UTC m=+106.940323498"
Sep 08 15:34:20.513378 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.513332    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-pkwt5" podStartSLOduration=33.853367906 podStartE2EDuration="36.513316196s" totalImagesPullingTime="2.65994829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:20.512340414 +0000 UTC m=+106.955532642" watchObservedRunningTime="2026-09-08 15:34:20.513316196 +0000 UTC m=+106.956508403"
Sep 08 15:34:20.652524 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.652483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bcdeb6e7-d979-4470-90e5-62fe8324dd90-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2hg4r\" (UID: \"bcdeb6e7-d979-4470-90e5-62fe8324dd90\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:20.655031 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.655006    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/bcdeb6e7-d979-4470-90e5-62fe8324dd90-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-2hg4r\" (UID: \"bcdeb6e7-d979-4470-90e5-62fe8324dd90\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:20.843406 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:20.843308    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:21.110807 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:21.110729    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"]
Sep 08 15:34:21.114261 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:21.114234    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcdeb6e7_d979_4470_90e5_62fe8324dd90.slice/crio-07c402254bb3f63aa4d9d8d336648ef7c0c42c99a684ce414c7aa275d399e7f6 WatchSource:0}: Error finding container 07c402254bb3f63aa4d9d8d336648ef7c0c42c99a684ce414c7aa275d399e7f6: Status 404 returned error can't find the container with id 07c402254bb3f63aa4d9d8d336648ef7c0c42c99a684ce414c7aa275d399e7f6
Sep 08 15:34:21.487112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:21.487073    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r" event={"ID":"bcdeb6e7-d979-4470-90e5-62fe8324dd90","Type":"ContainerStarted","Data":"07c402254bb3f63aa4d9d8d336648ef7c0c42c99a684ce414c7aa275d399e7f6"}
Sep 08 15:34:22.492478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:22.492443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r" event={"ID":"bcdeb6e7-d979-4470-90e5-62fe8324dd90","Type":"ContainerStarted","Data":"1ca4e58882ab0656f3c7a48f07723a659ca31a973a4bad9c21b4ab51b80630ab"}
Sep 08 15:34:22.492888 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:22.492611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:22.497011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:22.496988    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r"
Sep 08 15:34:22.507503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:22.507449    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-2hg4r" podStartSLOduration=2.511236389 podStartE2EDuration="3.507432701s" totalImagesPullingTime="996.196312ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:22.507026889 +0000 UTC m=+108.950219109" watchObservedRunningTime="2026-09-08 15:34:22.507432701 +0000 UTC m=+108.950624912"
Sep 08 15:34:23.052483 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.052443    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"]
Sep 08 15:34:23.054368 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.054349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.057716 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.057688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 08 15:34:23.057716 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.057693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p5d97\""
Sep 08 15:34:23.057941 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.057719    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 08 15:34:23.057941 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.057706    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 15:34:23.071701 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.071670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"]
Sep 08 15:34:23.171064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.171024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.171064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.171068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57aa1c97-2776-4750-9282-08498e43ae48-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.171316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.171089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj864\" (UniqueName: \"kubernetes.io/projected/57aa1c97-2776-4750-9282-08498e43ae48-kube-api-access-qj864\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.171316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.171117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.272336 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.272281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.272496 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.272347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57aa1c97-2776-4750-9282-08498e43ae48-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.272496 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.272367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj864\" (UniqueName: \"kubernetes.io/projected/57aa1c97-2776-4750-9282-08498e43ae48-kube-api-access-qj864\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.272496 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.272388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.272496 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:23.272419    2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 08 15:34:23.272696 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:23.272501    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-tls podName:57aa1c97-2776-4750-9282-08498e43ae48 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:23.772477794 +0000 UTC m=+110.215669988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-rmbj5" (UID: "57aa1c97-2776-4750-9282-08498e43ae48") : secret "prometheus-operator-tls" not found
Sep 08 15:34:23.273101 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.273077    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57aa1c97-2776-4750-9282-08498e43ae48-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.274804 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.274777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.281930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.281906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj864\" (UniqueName: \"kubernetes.io/projected/57aa1c97-2776-4750-9282-08498e43ae48-kube-api-access-qj864\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.776701 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.776660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.779139 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.779107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/57aa1c97-2776-4750-9282-08498e43ae48-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-rmbj5\" (UID: \"57aa1c97-2776-4750-9282-08498e43ae48\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:23.963684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:23.963639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"
Sep 08 15:34:24.236992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:24.236956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-rmbj5"]
Sep 08 15:34:24.240451 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:24.240420    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57aa1c97_2776_4750_9282_08498e43ae48.slice/crio-b45fdd7bf8f215c92d8240d6a844913f0f0913a7277b4777533b45aa8c99b98b WatchSource:0}: Error finding container b45fdd7bf8f215c92d8240d6a844913f0f0913a7277b4777533b45aa8c99b98b: Status 404 returned error can't find the container with id b45fdd7bf8f215c92d8240d6a844913f0f0913a7277b4777533b45aa8c99b98b
Sep 08 15:34:24.501020 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:24.500931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5" event={"ID":"57aa1c97-2776-4750-9282-08498e43ae48","Type":"ContainerStarted","Data":"b45fdd7bf8f215c92d8240d6a844913f0f0913a7277b4777533b45aa8c99b98b"}
Sep 08 15:34:25.506473 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:25.506438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5" event={"ID":"57aa1c97-2776-4750-9282-08498e43ae48","Type":"ContainerStarted","Data":"0daf4d146bd1df9e37653f6d705c6a1eda91497f4a480e001ee13a547401e227"}
Sep 08 15:34:25.506864 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:25.506492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5" event={"ID":"57aa1c97-2776-4750-9282-08498e43ae48","Type":"ContainerStarted","Data":"5dfab3b9074e008a0e3b6bd4aaa5df0321fac9c64509478180ac67ec1a830714"}
Sep 08 15:34:25.522156 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:25.522101    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-rmbj5" podStartSLOduration=1.429996682 podStartE2EDuration="2.522086107s" totalImagesPullingTime="1.092089425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:25.52156239 +0000 UTC m=+111.964754610" watchObservedRunningTime="2026-09-08 15:34:25.522086107 +0000 UTC m=+111.965278316"
Sep 08 15:34:26.473170 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:26.473140    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zvvx9"
Sep 08 15:34:27.425370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.425333    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gsdjv"]
Sep 08 15:34:27.427775 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.427755    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.431070 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.431048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 15:34:27.431378 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.431354    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 15:34:27.431528 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.431512    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gfwpq\""
Sep 08 15:34:27.439489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.439465    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 15:34:27.503143 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-wtmp\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503143 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-textfile\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503180    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbx6t\" (UniqueName: \"kubernetes.io/projected/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-kube-api-access-fbx6t\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-root\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-accelerators-collector-config\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-tls\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-sys\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.503607 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.503444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-metrics-client-ca\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.603956 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.603917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-wtmp\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.603956 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.603955    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-textfile\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.603985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbx6t\" (UniqueName: \"kubernetes.io/projected/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-kube-api-access-fbx6t\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-root\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-accelerators-collector-config\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-tls\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-sys\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-wtmp\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-root\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604585 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-sys\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604585 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-metrics-client-ca\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604585 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-textfile\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604744 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-metrics-client-ca\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.604792 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.604734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-accelerators-collector-config\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.606554 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.606534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-tls\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.606870 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.606586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.613729 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.613706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbx6t\" (UniqueName: \"kubernetes.io/projected/9173d83b-f216-4aed-9c2f-e7e19f9c86e3-kube-api-access-fbx6t\") pod \"node-exporter-gsdjv\" (UID: \"9173d83b-f216-4aed-9c2f-e7e19f9c86e3\") " pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.737340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:27.737236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gsdjv"
Sep 08 15:34:27.749095 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:27.749069    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9173d83b_f216_4aed_9c2f_e7e19f9c86e3.slice/crio-08f13b8f26a7a394219558ba52480efeeda291fa121d02bb93f877115187d6fd WatchSource:0}: Error finding container 08f13b8f26a7a394219558ba52480efeeda291fa121d02bb93f877115187d6fd: Status 404 returned error can't find the container with id 08f13b8f26a7a394219558ba52480efeeda291fa121d02bb93f877115187d6fd
Sep 08 15:34:28.507045 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.507004    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:34:28.510002 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.509977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.512587 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.512560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 15:34:28.512724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.512560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 15:34:28.512984 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.512966    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 15:34:28.513109 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513091    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 15:34:28.513182 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513168    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 15:34:28.513261 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513249    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 15:34:28.513380 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513366    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 15:34:28.513438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513396    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 15:34:28.513674 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 15:34:28.513793 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.513681    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tn4j2\""
Sep 08 15:34:28.520961 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.520932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gsdjv" event={"ID":"9173d83b-f216-4aed-9c2f-e7e19f9c86e3","Type":"ContainerStarted","Data":"08f13b8f26a7a394219558ba52480efeeda291fa121d02bb93f877115187d6fd"}
Sep 08 15:34:28.529388 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.529275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.613972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-out\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnplq\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-kube-api-access-rnplq\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-volume\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-web-config\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.614905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.614499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.714954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-volume\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-web-config\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-out\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnplq\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-kube-api-access-rnplq\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.715370    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.715848 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:28.715699    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 08 15:34:28.715848 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:28.715773    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls podName:ea8e4f56-668f-412d-b90f-8a35ccbba2a8 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:29.215752265 +0000 UTC m=+115.658944473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8") : secret "alertmanager-main-tls" not found
Sep 08 15:34:28.718555 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:28.718510    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle podName:ea8e4f56-668f-412d-b90f-8a35ccbba2a8 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:29.218488449 +0000 UTC m=+115.661680655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8") : configmap references non-existent config key: ca-bundle.crt
Sep 08 15:34:28.718718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.718622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.719795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.718904    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.719795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.719088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.719795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.719151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.719795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.719618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.719795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.719639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.720522 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.720481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-volume\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.720617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.720581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.721560 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.721536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-out\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.722079 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.722060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-web-config\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:28.727188 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:28.727138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnplq\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-kube-api-access-rnplq\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:29.219375 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.219326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:29.219663 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.219466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:29.220304 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.220260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:29.222014 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.221989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:29.422896 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.422855    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:34:29.526370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.526332    2563 generic.go:356] "Generic (PLEG): container finished" podID="9173d83b-f216-4aed-9c2f-e7e19f9c86e3" containerID="cd02cd082fb51188edb6bdced2f5316020addc4a5d4939c9019173b97dae524f" exitCode=0
Sep 08 15:34:29.526806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.526381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gsdjv" event={"ID":"9173d83b-f216-4aed-9c2f-e7e19f9c86e3","Type":"ContainerDied","Data":"cd02cd082fb51188edb6bdced2f5316020addc4a5d4939c9019173b97dae524f"}
Sep 08 15:34:29.714502 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:29.714464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:34:29.718891 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:29.718861    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8e4f56_668f_412d_b90f_8a35ccbba2a8.slice/crio-6fa192a57e4567c49bce49fa16063513c3b2e15d5e9bd449a6d3e1ab6fc904a6 WatchSource:0}: Error finding container 6fa192a57e4567c49bce49fa16063513c3b2e15d5e9bd449a6d3e1ab6fc904a6: Status 404 returned error can't find the container with id 6fa192a57e4567c49bce49fa16063513c3b2e15d5e9bd449a6d3e1ab6fc904a6
Sep 08 15:34:30.538081 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:30.537939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"6fa192a57e4567c49bce49fa16063513c3b2e15d5e9bd449a6d3e1ab6fc904a6"}
Sep 08 15:34:30.541012 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:30.540981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gsdjv" event={"ID":"9173d83b-f216-4aed-9c2f-e7e19f9c86e3","Type":"ContainerStarted","Data":"7627e1e8bf7f8eac490bcdc9639c177012460ac49c2dc4ad982c81d1923441ab"}
Sep 08 15:34:30.541166 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:30.541035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gsdjv" event={"ID":"9173d83b-f216-4aed-9c2f-e7e19f9c86e3","Type":"ContainerStarted","Data":"8bc53bb71c18135db88e2961e923437a31fcf79199e80942acaf8f4b677f2aa3"}
Sep 08 15:34:30.565582 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:30.565514    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gsdjv" podStartSLOduration=2.410645222 podStartE2EDuration="3.565492984s" totalImagesPullingTime="1.088532636s" totalInitContainerRuntime="66.315126ms" isStatefulPod=true podCreationTimestamp="2026-09-08 15:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:30.564918435 +0000 UTC m=+117.008110672" watchObservedRunningTime="2026-09-08 15:34:30.565492984 +0000 UTC m=+117.008685194"
Sep 08 15:34:31.547515 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:31.547478    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43" exitCode=0
Sep 08 15:34:31.547904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:31.547567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43"}
Sep 08 15:34:32.216505 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.216466    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"]
Sep 08 15:34:32.218898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.218875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.221424 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.221398    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dnf5r\""
Sep 08 15:34:32.221752 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.221604    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 08 15:34:32.227689 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.227655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"]
Sep 08 15:34:32.344951 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.344911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x98lj\" (UniqueName: \"kubernetes.io/projected/72959bd5-6be2-4c81-a585-02777b50fff6-kube-api-access-x98lj\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.345119 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.344978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72959bd5-6be2-4c81-a585-02777b50fff6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.446155 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.446120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x98lj\" (UniqueName: \"kubernetes.io/projected/72959bd5-6be2-4c81-a585-02777b50fff6-kube-api-access-x98lj\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.446382 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.446188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72959bd5-6be2-4c81-a585-02777b50fff6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.446382 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:32.446329    2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 08 15:34:32.446527 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:34:32.446399    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72959bd5-6be2-4c81-a585-02777b50fff6-monitoring-plugin-cert podName:72959bd5-6be2-4c81-a585-02777b50fff6 nodeName:}" failed. No retries permitted until 2026-09-08 15:34:32.946377602 +0000 UTC m=+119.389569792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/72959bd5-6be2-4c81-a585-02777b50fff6-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-pjm68" (UID: "72959bd5-6be2-4c81-a585-02777b50fff6") : secret "monitoring-plugin-cert" not found
Sep 08 15:34:32.460467 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.460439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x98lj\" (UniqueName: \"kubernetes.io/projected/72959bd5-6be2-4c81-a585-02777b50fff6-kube-api-access-x98lj\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.699920 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.699882    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"]
Sep 08 15:34:32.702246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.702222    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.705064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.704887    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 08 15:34:32.705064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.704898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-csrhv\""
Sep 08 15:34:32.705064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.704994    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 08 15:34:32.705064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.705041    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 08 15:34:32.705366 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.705145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 08 15:34:32.705366 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.705362    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 08 15:34:32.711230 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.710935    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 08 15:34:32.718017 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.717993    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"]
Sep 08 15:34:32.749354 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-metrics-client-ca\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-serving-certs-ca-bundle\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzbls\" (UniqueName: \"kubernetes.io/projected/c1e87551-aeea-4cb2-9473-98f5c1491e9e-kube-api-access-kzbls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749671 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-secret-telemeter-client\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749671 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-federate-client-tls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749671 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749641    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.749817 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.749680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-telemeter-client-tls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.850719 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.850674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-metrics-client-ca\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.850866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.850742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-serving-certs-ca-bundle\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.850866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.850787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.850866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.850807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzbls\" (UniqueName: \"kubernetes.io/projected/c1e87551-aeea-4cb2-9473-98f5c1491e9e-kube-api-access-kzbls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.850866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.850829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-secret-telemeter-client\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.850866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.850847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-federate-client-tls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.851168 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.851135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.851234 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.851199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-telemeter-client-tls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.851653 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.851622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-metrics-client-ca\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.851768 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.851622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-serving-certs-ca-bundle\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.852316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.852249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1e87551-aeea-4cb2-9473-98f5c1491e9e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.854519 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.854479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.854676 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.854649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-secret-telemeter-client\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.855007 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.854986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-federate-client-tls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.855594 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.855547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c1e87551-aeea-4cb2-9473-98f5c1491e9e-telemeter-client-tls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.863382 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.863351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzbls\" (UniqueName: \"kubernetes.io/projected/c1e87551-aeea-4cb2-9473-98f5c1491e9e-kube-api-access-kzbls\") pod \"telemeter-client-6b95879bfd-mn6hd\" (UID: \"c1e87551-aeea-4cb2-9473-98f5c1491e9e\") " pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:32.954273 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.953616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72959bd5-6be2-4c81-a585-02777b50fff6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:32.957305 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:32.957256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72959bd5-6be2-4c81-a585-02777b50fff6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pjm68\" (UID: \"72959bd5-6be2-4c81-a585-02777b50fff6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:33.014632 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.014597    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"
Sep 08 15:34:33.132369 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.132129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:33.315777 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.315751    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b95879bfd-mn6hd"]
Sep 08 15:34:33.318428 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:33.318402    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e87551_aeea_4cb2_9473_98f5c1491e9e.slice/crio-f541fb28b54a11c80b46b536f902587c2ba2e2f177ab1d30ddd1bd840815bd60 WatchSource:0}: Error finding container f541fb28b54a11c80b46b536f902587c2ba2e2f177ab1d30ddd1bd840815bd60: Status 404 returned error can't find the container with id f541fb28b54a11c80b46b536f902587c2ba2e2f177ab1d30ddd1bd840815bd60
Sep 08 15:34:33.410066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.410026    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"]
Sep 08 15:34:33.413758 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:33.413709    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72959bd5_6be2_4c81_a585_02777b50fff6.slice/crio-17a4075bc2283892ae303f43ec6262b3fb2eb11c804ce78787221ff20e2b5251 WatchSource:0}: Error finding container 17a4075bc2283892ae303f43ec6262b3fb2eb11c804ce78787221ff20e2b5251: Status 404 returned error can't find the container with id 17a4075bc2283892ae303f43ec6262b3fb2eb11c804ce78787221ff20e2b5251
Sep 08 15:34:33.561228 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.561130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca"}
Sep 08 15:34:33.561228 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.561201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c"}
Sep 08 15:34:33.561489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.561246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52"}
Sep 08 15:34:33.561489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.561273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8"}
Sep 08 15:34:33.561489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.561330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795"}
Sep 08 15:34:33.562510 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.562482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd" event={"ID":"c1e87551-aeea-4cb2-9473-98f5c1491e9e","Type":"ContainerStarted","Data":"f541fb28b54a11c80b46b536f902587c2ba2e2f177ab1d30ddd1bd840815bd60"}
Sep 08 15:34:33.563689 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.563663    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68" event={"ID":"72959bd5-6be2-4c81-a585-02777b50fff6","Type":"ContainerStarted","Data":"17a4075bc2283892ae303f43ec6262b3fb2eb11c804ce78787221ff20e2b5251"}
Sep 08 15:34:33.741833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.741791    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:34:33.745039 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.745006    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.747568 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.747539    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 15:34:33.747721 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.747656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 15:34:33.747833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.747784    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 15:34:33.747995 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.747979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 15:34:33.748079 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.748060    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 15:34:33.748161 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.748127    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 15:34:33.748220 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.748128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2kecbjccmih1l\""
Sep 08 15:34:33.749140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.748875    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 15:34:33.749140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.748904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 15:34:33.749140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.748924    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 15:34:33.749140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.749002    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 15:34:33.749511 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.749225    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 15:34:33.749511 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.749348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-djg2d\""
Sep 08 15:34:33.749511 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.749377    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 15:34:33.749669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.749646    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 15:34:33.749829 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.749646    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 15:34:33.751238 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.751213    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 15:34:33.775514 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.775442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:34:33.862121 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862121 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862121 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862121 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862402 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5f99\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-kube-api-access-l5f99\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862766 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862766 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.862766 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.862653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.963859 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.963817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.963930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.963990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5f99\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-kube-api-access-l5f99\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.964481 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964942    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.964983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.965267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.965904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.965564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.971992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.968830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.973365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.973057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.974809 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.974762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.975961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.976398    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.976621    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.976855    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.977119    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.977607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.977862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.978487    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.979241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5f99\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-kube-api-access-l5f99\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.980149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.979761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.984762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.981525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.984762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.981933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.984762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.984572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.985939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.985899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.986444 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.986422    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 15:34:33.987861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.987827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:33.987861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:33.987846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:34.002586 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.002553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:34.061235 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.061205    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-djg2d\""
Sep 08 15:34:34.069464 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.069435    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:34.377732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.377687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:34:34.383063 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:34.383032    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b1ba2e8_ed7e_4834_8f27_cac9836b466c.slice/crio-ae187abd9b3a2c0226c5fa66389e6f5e46b2e016436e3bdd390bc5f896b106d5 WatchSource:0}: Error finding container ae187abd9b3a2c0226c5fa66389e6f5e46b2e016436e3bdd390bc5f896b106d5: Status 404 returned error can't find the container with id ae187abd9b3a2c0226c5fa66389e6f5e46b2e016436e3bdd390bc5f896b106d5
Sep 08 15:34:34.570389 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.570348    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085" exitCode=0
Sep 08 15:34:34.570555 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.570437    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085"}
Sep 08 15:34:34.570617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.570588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"ae187abd9b3a2c0226c5fa66389e6f5e46b2e016436e3bdd390bc5f896b106d5"}
Sep 08 15:34:34.574617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.574594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerStarted","Data":"39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e"}
Sep 08 15:34:34.632339 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:34.632267    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.037630139 podStartE2EDuration="6.632242418s" totalImagesPullingTime="3.578251398s" totalInitContainerRuntime="16.360881ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:34.630444479 +0000 UTC m=+121.073636712" watchObservedRunningTime="2026-09-08 15:34:34.632242418 +0000 UTC m=+121.075434628"
Sep 08 15:34:36.586704 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.586657    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd" event={"ID":"c1e87551-aeea-4cb2-9473-98f5c1491e9e","Type":"ContainerStarted","Data":"6b80c6df6c102d81cca5eda51472405e9c0cabab5234852aad513fd2f9626590"}
Sep 08 15:34:36.587196 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.586737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd" event={"ID":"c1e87551-aeea-4cb2-9473-98f5c1491e9e","Type":"ContainerStarted","Data":"be135dc6a4e5a95ddd96ff6eb026997a79bb731c2ff57c5b09a030fca6bf81f7"}
Sep 08 15:34:36.587196 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.586766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd" event={"ID":"c1e87551-aeea-4cb2-9473-98f5c1491e9e","Type":"ContainerStarted","Data":"d372c38d45bf8714de7cdf3a3bd3342637363a1c0feec234c3c7b40ebf489fbe"}
Sep 08 15:34:36.588691 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.588658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68" event={"ID":"72959bd5-6be2-4c81-a585-02777b50fff6","Type":"ContainerStarted","Data":"02a9a1beba43cd983af1a3e3436dd94eed9fb4de375c1bdc301dad9cc575d222"}
Sep 08 15:34:36.588889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.588870    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:36.594607 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.594577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68"
Sep 08 15:34:36.620544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.620483    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b95879bfd-mn6hd" podStartSLOduration=2.3803291890000002 podStartE2EDuration="4.620465552s" totalImagesPullingTime="2.240136363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:36.620160944 +0000 UTC m=+123.063353196" watchObservedRunningTime="2026-09-08 15:34:36.620465552 +0000 UTC m=+123.063657762"
Sep 08 15:34:36.649385 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:36.649330    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pjm68" podStartSLOduration=2.508183163 podStartE2EDuration="4.649312507s" totalImagesPullingTime="2.141129344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:36.647544192 +0000 UTC m=+123.090736423" watchObservedRunningTime="2026-09-08 15:34:36.649312507 +0000 UTC m=+123.092504718"
Sep 08 15:34:38.605328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:38.605267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74"}
Sep 08 15:34:38.605801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:38.605385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea"}
Sep 08 15:34:40.616612 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:40.616573    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098"}
Sep 08 15:34:40.617006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:40.616650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2"}
Sep 08 15:34:40.617006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:40.616677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853"}
Sep 08 15:34:40.617006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:40.616709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerStarted","Data":"d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506"}
Sep 08 15:34:40.654529 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:40.654464    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.68664174 podStartE2EDuration="7.65444169s" totalImagesPullingTime="4.944162633s" totalInitContainerRuntime="23.637317ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:34:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:40.652395393 +0000 UTC m=+127.095587630" watchObservedRunningTime="2026-09-08 15:34:40.65444169 +0000 UTC m=+127.097633900"
Sep 08 15:34:43.870047 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:43.870001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:34:43.872334 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:43.872305    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3bdb3868-cedc-47d0-9c6c-cadb57a2b95e-metrics-certs\") pod \"network-metrics-daemon-dxkzl\" (UID: \"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e\") " pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:34:44.010559 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:44.010521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xsqld\""
Sep 08 15:34:44.019269 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:44.019244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dxkzl"
Sep 08 15:34:44.071687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:44.071571    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:34:44.291198 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:44.291165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dxkzl"]
Sep 08 15:34:44.293729 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:34:44.293698    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bdb3868_cedc_47d0_9c6c_cadb57a2b95e.slice/crio-f5b8864a98091e36e83064c47e2829afd1adf22b8587ace1fdf132f7fcb52fa0 WatchSource:0}: Error finding container f5b8864a98091e36e83064c47e2829afd1adf22b8587ace1fdf132f7fcb52fa0: Status 404 returned error can't find the container with id f5b8864a98091e36e83064c47e2829afd1adf22b8587ace1fdf132f7fcb52fa0
Sep 08 15:34:44.633063 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:44.632973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxkzl" event={"ID":"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e","Type":"ContainerStarted","Data":"f5b8864a98091e36e83064c47e2829afd1adf22b8587ace1fdf132f7fcb52fa0"}
Sep 08 15:34:45.638606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:45.638508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxkzl" event={"ID":"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e","Type":"ContainerStarted","Data":"f0cc1043d820c53357e184c46e8ac2f56828a50ec3a2906e5a4b97ee08ce3bfb"}
Sep 08 15:34:45.638606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:45.638570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dxkzl" event={"ID":"3bdb3868-cedc-47d0-9c6c-cadb57a2b95e","Type":"ContainerStarted","Data":"0a8c329a9972f65a343819d6f0d883e6b809addec7b0c70021265268bf1e2918"}
Sep 08 15:34:45.659516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:34:45.659462    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dxkzl" podStartSLOduration=129.671416497 podStartE2EDuration="2m10.65944244s" totalImagesPullingTime="988.025943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:34:45.658917426 +0000 UTC m=+132.102109646" watchObservedRunningTime="2026-09-08 15:34:45.65944244 +0000 UTC m=+132.102634650"
Sep 08 15:35:03.709387 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:03.709349    2563 generic.go:356] "Generic (PLEG): container finished" podID="bad73a4b-a2fd-4520-ad6d-76467cc01927" containerID="83769695a2819ef189e8f262d8c7ce0713fe73a32858a8f9743acd7c8f753512" exitCode=0
Sep 08 15:35:03.709823 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:03.709421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh" event={"ID":"bad73a4b-a2fd-4520-ad6d-76467cc01927","Type":"ContainerDied","Data":"83769695a2819ef189e8f262d8c7ce0713fe73a32858a8f9743acd7c8f753512"}
Sep 08 15:35:03.709823 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:03.709762    2563 scope.go:117] "RemoveContainer" containerID="83769695a2819ef189e8f262d8c7ce0713fe73a32858a8f9743acd7c8f753512"
Sep 08 15:35:04.715428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:04.715391    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-tbbgh" event={"ID":"bad73a4b-a2fd-4520-ad6d-76467cc01927","Type":"ContainerStarted","Data":"c32666dab78d8ddb61626c015d41d803d3d7398abed046b069fa720f6d1f7235"}
Sep 08 15:35:34.072172 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:34.072073    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:34.087855 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:34.087827    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:34.855331 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:34.855298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:47.903526 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.903491    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:35:47.904023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.903792    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="alertmanager" containerID="cri-o://29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795" gracePeriod=120
Sep 08 15:35:47.904023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.903840    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-metric" containerID="cri-o://d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca" gracePeriod=120
Sep 08 15:35:47.904023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.903852    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-web" containerID="cri-o://1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52" gracePeriod=120
Sep 08 15:35:47.904023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.903863    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy" containerID="cri-o://84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c" gracePeriod=120
Sep 08 15:35:47.904023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.903913    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="config-reloader" containerID="cri-o://d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8" gracePeriod=120
Sep 08 15:35:47.904321 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:47.904017    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="prom-label-proxy" containerID="cri-o://39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e" gracePeriod=120
Sep 08 15:35:48.897553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897520    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e" exitCode=0
Sep 08 15:35:48.897553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897546    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca" exitCode=0
Sep 08 15:35:48.897553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897553    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c" exitCode=0
Sep 08 15:35:48.897553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897559    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8" exitCode=0
Sep 08 15:35:48.897553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897564    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795" exitCode=0
Sep 08 15:35:48.897839 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e"}
Sep 08 15:35:48.897839 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca"}
Sep 08 15:35:48.897839 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c"}
Sep 08 15:35:48.897839 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8"}
Sep 08 15:35:48.897839 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:48.897737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795"}
Sep 08 15:35:49.152221 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.152164    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:49.259448 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259418    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.259621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-metrics-client-ca\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.259621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.259621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259518    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.259621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259537    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnplq\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-kube-api-access-rnplq\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.259621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259559    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-tls-assets\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.259621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259574    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-web-config\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.259970    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:49.260091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-out\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260092    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-web\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260124    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-cluster-tls-config\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-main-db\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260378 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-volume\") pod \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\" (UID: \"ea8e4f56-668f-412d-b90f-8a35ccbba2a8\") "
Sep 08 15:35:49.260825 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.260549    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-metrics-client-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.262281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.262196    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.262553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.262527    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:49.262645 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.262595    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:49.262951 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.262916    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.263053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.262996    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.263225 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.263171    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:49.263377 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.263354    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-kube-api-access-rnplq" (OuterVolumeSpecName: "kube-api-access-rnplq") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "kube-api-access-rnplq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:49.263543 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.263521    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-volume" (OuterVolumeSpecName: "config-volume") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.264471 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.264451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-out" (OuterVolumeSpecName: "config-out") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:49.264955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.264930    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.267182 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.267103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.273894 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.273865    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-web-config" (OuterVolumeSpecName: "web-config") pod "ea8e4f56-668f-412d-b90f-8a35ccbba2a8" (UID: "ea8e4f56-668f-412d-b90f-8a35ccbba2a8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:49.361906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361875    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-main-db\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.361906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361902    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.361906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361912    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.361906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361922    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-main-tls\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361932    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361941    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnplq\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-kube-api-access-rnplq\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361949    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-tls-assets\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361958    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-web-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361965    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-config-out\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361973    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361982    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-cluster-tls-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.362169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.361991    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8e4f56-668f-412d-b90f-8a35ccbba2a8-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:49.905078 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.905041    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerID="1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52" exitCode=0
Sep 08 15:35:49.905247 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.905114    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:49.905247 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.905125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52"}
Sep 08 15:35:49.905247 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.905191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea8e4f56-668f-412d-b90f-8a35ccbba2a8","Type":"ContainerDied","Data":"6fa192a57e4567c49bce49fa16063513c3b2e15d5e9bd449a6d3e1ab6fc904a6"}
Sep 08 15:35:49.905247 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.905225    2563 scope.go:117] "RemoveContainer" containerID="39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e"
Sep 08 15:35:49.913234 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.913215    2563 scope.go:117] "RemoveContainer" containerID="d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca"
Sep 08 15:35:49.921271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.921251    2563 scope.go:117] "RemoveContainer" containerID="84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c"
Sep 08 15:35:49.928278 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.928259    2563 scope.go:117] "RemoveContainer" containerID="1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52"
Sep 08 15:35:49.931930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.931905    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:35:49.936066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.936042    2563 scope.go:117] "RemoveContainer" containerID="d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8"
Sep 08 15:35:49.938463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.938443    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:35:49.943795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.943777    2563 scope.go:117] "RemoveContainer" containerID="29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795"
Sep 08 15:35:49.951073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.951056    2563 scope.go:117] "RemoveContainer" containerID="5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43"
Sep 08 15:35:49.958211 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.958193    2563 scope.go:117] "RemoveContainer" containerID="39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e"
Sep 08 15:35:49.958531 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.958504    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e\": container with ID starting with 39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e not found: ID does not exist" containerID="39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e"
Sep 08 15:35:49.958589 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.958544    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e"} err="failed to get container status \"39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e\": rpc error: code = NotFound desc = could not find container \"39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e\": container with ID starting with 39796072ccadca33e7983150729b64da0c231a554c48b612616f141280b52f1e not found: ID does not exist"
Sep 08 15:35:49.958589 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.958580    2563 scope.go:117] "RemoveContainer" containerID="d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca"
Sep 08 15:35:49.958807 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.958791    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca\": container with ID starting with d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca not found: ID does not exist" containerID="d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca"
Sep 08 15:35:49.958851 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.958811    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca"} err="failed to get container status \"d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca\": rpc error: code = NotFound desc = could not find container \"d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca\": container with ID starting with d3c66763adf71f1cc24e39c16072f1deef7597b1f015057ef7aaf8f090383bca not found: ID does not exist"
Sep 08 15:35:49.958851 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.958826    2563 scope.go:117] "RemoveContainer" containerID="84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c"
Sep 08 15:35:49.959083 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.959067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c\": container with ID starting with 84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c not found: ID does not exist" containerID="84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c"
Sep 08 15:35:49.959130 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959085    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c"} err="failed to get container status \"84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c\": rpc error: code = NotFound desc = could not find container \"84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c\": container with ID starting with 84746e208e9a69f4c1b4959dba02da4da415f6934c2ca7266e1adc3b1b924a8c not found: ID does not exist"
Sep 08 15:35:49.959130 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959102    2563 scope.go:117] "RemoveContainer" containerID="1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52"
Sep 08 15:35:49.959322 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.959304    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52\": container with ID starting with 1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52 not found: ID does not exist" containerID="1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52"
Sep 08 15:35:49.959360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959331    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52"} err="failed to get container status \"1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52\": rpc error: code = NotFound desc = could not find container \"1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52\": container with ID starting with 1eb48271ab4ddf6ae8d3e69b07643d4bce27d2cca8d1ce27d3a5cb2a8da9de52 not found: ID does not exist"
Sep 08 15:35:49.959360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959350    2563 scope.go:117] "RemoveContainer" containerID="d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8"
Sep 08 15:35:49.959578 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.959559    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8\": container with ID starting with d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8 not found: ID does not exist" containerID="d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8"
Sep 08 15:35:49.959617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959584    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8"} err="failed to get container status \"d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8\": rpc error: code = NotFound desc = could not find container \"d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8\": container with ID starting with d2fb3bab8ed5bae660baa994d86073c97a687cde4496a91e51dae457d7806fa8 not found: ID does not exist"
Sep 08 15:35:49.959617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959603    2563 scope.go:117] "RemoveContainer" containerID="29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795"
Sep 08 15:35:49.959809 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.959792    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795\": container with ID starting with 29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795 not found: ID does not exist" containerID="29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795"
Sep 08 15:35:49.959848 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959813    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795"} err="failed to get container status \"29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795\": rpc error: code = NotFound desc = could not find container \"29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795\": container with ID starting with 29e228912ded4da3d1cd9b71e35678b8902344499f8280c1718a02b0776e8795 not found: ID does not exist"
Sep 08 15:35:49.959848 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.959826    2563 scope.go:117] "RemoveContainer" containerID="5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43"
Sep 08 15:35:49.960018 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:49.960004    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43\": container with ID starting with 5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43 not found: ID does not exist" containerID="5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43"
Sep 08 15:35:49.960052 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.960020    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43"} err="failed to get container status \"5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43\": rpc error: code = NotFound desc = could not find container \"5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43\": container with ID starting with 5e0884b49a379414bfcbd4179dd8605af243cc166dc7c4267aacb604c37bfa43 not found: ID does not exist"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974027    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974699    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="init-config-reloader"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974720    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="init-config-reloader"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974745    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974755    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974765    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="config-reloader"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974775    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="config-reloader"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974794    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="alertmanager"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974803    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="alertmanager"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974832    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="prom-label-proxy"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974840    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="prom-label-proxy"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974868    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-web"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974877    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-web"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974887    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-metric"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.974896    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-metric"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.975033    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="config-reloader"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.975045    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.975056    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-web"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.975070    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="prom-label-proxy"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.975081    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="alertmanager"
Sep 08 15:35:49.976534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.975095    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" containerName="kube-rbac-proxy-metric"
Sep 08 15:35:49.981640 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.981612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:49.984302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984165    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 15:35:49.984302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 15:35:49.984302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984192    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 15:35:49.984302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984198    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 15:35:49.984302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984223    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 15:35:49.984302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984231    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 15:35:49.984774 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984683    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 15:35:49.984774 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.984690    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 15:35:49.985177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.985159    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tn4j2\""
Sep 08 15:35:49.990034 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.990010    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 15:35:49.992087 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:49.992063    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:35:50.067112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-web-config\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7da9185-fcba-46dd-aad2-a7b588bfbde7-config-out\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7da9185-fcba-46dd-aad2-a7b588bfbde7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7da9185-fcba-46dd-aad2-a7b588bfbde7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7da9185-fcba-46dd-aad2-a7b588bfbde7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7da9185-fcba-46dd-aad2-a7b588bfbde7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nrd7\" (UniqueName: \"kubernetes.io/projected/d7da9185-fcba-46dd-aad2-a7b588bfbde7-kube-api-access-7nrd7\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.067569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.067533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.099165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.099133    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8e4f56-668f-412d-b90f-8a35ccbba2a8" path="/var/lib/kubelet/pods/ea8e4f56-668f-412d-b90f-8a35ccbba2a8/volumes"
Sep 08 15:35:50.168788 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.168788 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.168788 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7da9185-fcba-46dd-aad2-a7b588bfbde7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7da9185-fcba-46dd-aad2-a7b588bfbde7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7da9185-fcba-46dd-aad2-a7b588bfbde7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168920    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7da9185-fcba-46dd-aad2-a7b588bfbde7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nrd7\" (UniqueName: \"kubernetes.io/projected/d7da9185-fcba-46dd-aad2-a7b588bfbde7-kube-api-access-7nrd7\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.168983    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.169011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-web-config\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.169092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7da9185-fcba-46dd-aad2-a7b588bfbde7-config-out\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.169140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.169169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.169427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.169342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7da9185-fcba-46dd-aad2-a7b588bfbde7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.170024 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.169691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7da9185-fcba-46dd-aad2-a7b588bfbde7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.170331 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.170306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7da9185-fcba-46dd-aad2-a7b588bfbde7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172123 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.171989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172123 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172001    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7da9185-fcba-46dd-aad2-a7b588bfbde7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172566 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-web-config\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172838 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7da9185-fcba-46dd-aad2-a7b588bfbde7-config-out\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.172872 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.173018 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.172997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.174078 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.174053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7da9185-fcba-46dd-aad2-a7b588bfbde7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.178372 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.178352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nrd7\" (UniqueName: \"kubernetes.io/projected/d7da9185-fcba-46dd-aad2-a7b588bfbde7-kube-api-access-7nrd7\") pod \"alertmanager-main-0\" (UID: \"d7da9185-fcba-46dd-aad2-a7b588bfbde7\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.297011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.296969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 15:35:50.587698 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.587669    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 15:35:50.589212 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:35:50.589183    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7da9185_fcba_46dd_aad2_a7b588bfbde7.slice/crio-5201dfa17d4f4490c33398c863c45aa708f2ab803b3238dbaf21dc578adb8ca3 WatchSource:0}: Error finding container 5201dfa17d4f4490c33398c863c45aa708f2ab803b3238dbaf21dc578adb8ca3: Status 404 returned error can't find the container with id 5201dfa17d4f4490c33398c863c45aa708f2ab803b3238dbaf21dc578adb8ca3
Sep 08 15:35:50.912164 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.912074    2563 generic.go:356] "Generic (PLEG): container finished" podID="d7da9185-fcba-46dd-aad2-a7b588bfbde7" containerID="8294bcf4ba3e177bca346e4bbbc4e339ba18f27a7d92aba83ef8e69ba9c1de93" exitCode=0
Sep 08 15:35:50.912164 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.912146    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerDied","Data":"8294bcf4ba3e177bca346e4bbbc4e339ba18f27a7d92aba83ef8e69ba9c1de93"}
Sep 08 15:35:50.912381 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:50.912201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"5201dfa17d4f4490c33398c863c45aa708f2ab803b3238dbaf21dc578adb8ca3"}
Sep 08 15:35:51.921385 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.921345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"f1e2c0c40bfb02a465790bdcb085adbc0af8ea4af7720d8226c85d7397aa07e6"}
Sep 08 15:35:51.922006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.921967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"68dceac283fdc397170f1c14b4413b1db0fc070a576ab6e0b015e47fa971db29"}
Sep 08 15:35:51.922113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.922033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"37055ed29275f913c9baa72e513059999c5c2838ebaeb0d34ace09cc7d22e5ca"}
Sep 08 15:35:51.922113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.922076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"d1a01fb5fed5c891c1df1dc9b48a586e1d482ce0fad57a88c30a90ef641d98fa"}
Sep 08 15:35:51.922186 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.922115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"17e8b31ed23aff7a205c21259d419e2582a3de34ee410c919a9fb2c07a3c4d7a"}
Sep 08 15:35:51.922186 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.922156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7da9185-fcba-46dd-aad2-a7b588bfbde7","Type":"ContainerStarted","Data":"2aa4464ca226bf18132b3cbfe180b4a3cea402a184bab30015ee191fdc2aa530"}
Sep 08 15:35:51.960761 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:51.960704    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.944429138 podStartE2EDuration="2.960679216s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.250078ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:51.958822651 +0000 UTC m=+198.402014909" watchObservedRunningTime="2026-09-08 15:35:51.960679216 +0000 UTC m=+198.403871428"
Sep 08 15:35:52.133743 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.133700    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:35:52.134450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.134413    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="prometheus" containerID="cri-o://29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea" gracePeriod=600
Sep 08 15:35:52.135011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.134627    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy" containerID="cri-o://61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2" gracePeriod=600
Sep 08 15:35:52.135905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.134673    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098" gracePeriod=600
Sep 08 15:35:52.135905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.134667    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="thanos-sidecar" containerID="cri-o://d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506" gracePeriod=600
Sep 08 15:35:52.136033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.134702    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-web" containerID="cri-o://375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853" gracePeriod=600
Sep 08 15:35:52.136033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.134721    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="config-reloader" containerID="cri-o://74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74" gracePeriod=600
Sep 08 15:35:52.930049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930014    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098" exitCode=0
Sep 08 15:35:52.930049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930042    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2" exitCode=0
Sep 08 15:35:52.930049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930048    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506" exitCode=0
Sep 08 15:35:52.930049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930054    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74" exitCode=0
Sep 08 15:35:52.930049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930060    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea" exitCode=0
Sep 08 15:35:52.930756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098"}
Sep 08 15:35:52.930756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930148    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2"}
Sep 08 15:35:52.930756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506"}
Sep 08 15:35:52.930756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74"}
Sep 08 15:35:52.930756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:52.930228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea"}
Sep 08 15:35:53.387654 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.387629    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:53.501764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501676    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-db\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.501764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501716    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5f99\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-kube-api-access-l5f99\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.501764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501739    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-1\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.501764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-tls\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-2\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501816    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-trusted-ca-bundle\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501833    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-kube-rbac-proxy\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501854    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-serving-certs-ca-bundle\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501880    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-tls-assets\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-metrics-client-ca\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.501984    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-0\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-metrics-client-certs\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502038    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-web-config\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502109    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.502724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.502724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.502724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-grpc-tls\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-kubelet-serving-ca-bundle\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config-out\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.502868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502847    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-thanos-prometheus-http-client-file\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.503066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.502882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\" (UID: \"9b1ba2e8-ed7e-4834-8f27-cac9836b466c\") "
Sep 08 15:35:53.503225 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.503193    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.503225 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.503219    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.503373 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.503225    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:53.503373 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.503235    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.503701 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.503668    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.503959 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.503796    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.504128 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.504100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.504490 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.504468    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-kube-api-access-l5f99" (OuterVolumeSpecName: "kube-api-access-l5f99") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "kube-api-access-l5f99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:53.504786 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.504763    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:35:53.505333 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.505303    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.506173 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.506118    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.506269 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.506195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.506507 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.506394    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.506507 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.506461    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.506878 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.506853    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:35:53.506958 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.506873    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.507541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.507514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config" (OuterVolumeSpecName: "config") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.508065 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.508030    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.508451 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.508434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config-out" (OuterVolumeSpecName: "config-out") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:35:53.515708 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.515677    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-web-config" (OuterVolumeSpecName: "web-config") pod "9b1ba2e8-ed7e-4834-8f27-cac9836b466c" (UID: "9b1ba2e8-ed7e-4834-8f27-cac9836b466c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:35:53.604438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604402    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-metrics-client-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604430    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604441    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-metrics-client-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604451    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-web-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604459    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-grpc-tls\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604467    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604475    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604484    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-config-out\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604493    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604505    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604514    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-prometheus-k8s-db\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604523    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5f99\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-kube-api-access-l5f99\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604532    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604541    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-kube-rbac-proxy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604550    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604558    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-tls-assets\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.604718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.604566    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9b1ba2e8-ed7e-4834-8f27-cac9836b466c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:35:53.938398 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.938365    2563 generic.go:356] "Generic (PLEG): container finished" podID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerID="375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853" exitCode=0
Sep 08 15:35:53.938806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.938443    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:53.938806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.938444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853"}
Sep 08 15:35:53.938806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.938523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9b1ba2e8-ed7e-4834-8f27-cac9836b466c","Type":"ContainerDied","Data":"ae187abd9b3a2c0226c5fa66389e6f5e46b2e016436e3bdd390bc5f896b106d5"}
Sep 08 15:35:53.938806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.938558    2563 scope.go:117] "RemoveContainer" containerID="da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098"
Sep 08 15:35:53.946481 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.946458    2563 scope.go:117] "RemoveContainer" containerID="61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2"
Sep 08 15:35:53.954124 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.954103    2563 scope.go:117] "RemoveContainer" containerID="375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853"
Sep 08 15:35:53.961414 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.961386    2563 scope.go:117] "RemoveContainer" containerID="d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506"
Sep 08 15:35:53.963245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.963224    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:35:53.969604 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.969514    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:35:53.969681 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.969638    2563 scope.go:117] "RemoveContainer" containerID="74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74"
Sep 08 15:35:53.978366 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.978343    2563 scope.go:117] "RemoveContainer" containerID="29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea"
Sep 08 15:35:53.986186 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.986161    2563 scope.go:117] "RemoveContainer" containerID="eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085"
Sep 08 15:35:53.993632 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.993609    2563 scope.go:117] "RemoveContainer" containerID="da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098"
Sep 08 15:35:53.993938 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.993919    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098\": container with ID starting with da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098 not found: ID does not exist" containerID="da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098"
Sep 08 15:35:53.994068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.993953    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098"} err="failed to get container status \"da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098\": rpc error: code = NotFound desc = could not find container \"da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098\": container with ID starting with da6ff674ae25574d0c1581a3bbdc33bc817b1757a41adbed18c8e9a813a8f098 not found: ID does not exist"
Sep 08 15:35:53.994068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.993978    2563 scope.go:117] "RemoveContainer" containerID="61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2"
Sep 08 15:35:53.994068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994033    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:35:53.994271 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.994253    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2\": container with ID starting with 61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2 not found: ID does not exist" containerID="61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2"
Sep 08 15:35:53.994360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994276    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2"} err="failed to get container status \"61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2\": rpc error: code = NotFound desc = could not find container \"61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2\": container with ID starting with 61304d8149b329215b2bcaf09d2c01f2cfd4a7e312718e3234eab3e57884b5c2 not found: ID does not exist"
Sep 08 15:35:53.994360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994307    2563 scope.go:117] "RemoveContainer" containerID="375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853"
Sep 08 15:35:53.994435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994382    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="prometheus"
Sep 08 15:35:53.994435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994397    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="prometheus"
Sep 08 15:35:53.994435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994412    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-thanos"
Sep 08 15:35:53.994435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994421    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-thanos"
Sep 08 15:35:53.994435 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994433    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="thanos-sidecar"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994439    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="thanos-sidecar"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994446    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="config-reloader"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994454    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="config-reloader"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994468    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-web"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994478    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-web"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994491    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994496    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994504    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="init-config-reloader"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994509    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="init-config-reloader"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.994549    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853\": container with ID starting with 375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853 not found: ID does not exist" containerID="375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994565    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-web"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994574    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="config-reloader"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994582    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="prometheus"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994587    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994593    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="thanos-sidecar"
Sep 08 15:35:53.994606 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994599    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" containerName="kube-rbac-proxy-thanos"
Sep 08 15:35:53.995073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994567    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853"} err="failed to get container status \"375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853\": rpc error: code = NotFound desc = could not find container \"375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853\": container with ID starting with 375c3980d231ba85da66498e126b686937ea24653adc4dafe326425cb9e34853 not found: ID does not exist"
Sep 08 15:35:53.995073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994661    2563 scope.go:117] "RemoveContainer" containerID="d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506"
Sep 08 15:35:53.995073 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.994899    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506\": container with ID starting with d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506 not found: ID does not exist" containerID="d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506"
Sep 08 15:35:53.995073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994917    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506"} err="failed to get container status \"d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506\": rpc error: code = NotFound desc = could not find container \"d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506\": container with ID starting with d9296f329dad888138817bf0e19d14003c0ee902e875a48dd781ffe45f508506 not found: ID does not exist"
Sep 08 15:35:53.995073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.994929    2563 scope.go:117] "RemoveContainer" containerID="74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74"
Sep 08 15:35:53.995237 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.995176    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74\": container with ID starting with 74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74 not found: ID does not exist" containerID="74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74"
Sep 08 15:35:53.995237 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.995195    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74"} err="failed to get container status \"74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74\": rpc error: code = NotFound desc = could not find container \"74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74\": container with ID starting with 74035f0391b6c6b0a3cc46efeed93f0d5ba752e2f5577969cbf6333e95fc3e74 not found: ID does not exist"
Sep 08 15:35:53.995237 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.995206    2563 scope.go:117] "RemoveContainer" containerID="29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea"
Sep 08 15:35:53.995451 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.995436    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea\": container with ID starting with 29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea not found: ID does not exist" containerID="29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea"
Sep 08 15:35:53.995490 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.995457    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea"} err="failed to get container status \"29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea\": rpc error: code = NotFound desc = could not find container \"29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea\": container with ID starting with 29385684710e211d817402debd4e47c9db2fe0f992a9e09f9591265da7d2b5ea not found: ID does not exist"
Sep 08 15:35:53.995490 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.995475    2563 scope.go:117] "RemoveContainer" containerID="eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085"
Sep 08 15:35:53.995717 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:35:53.995704    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085\": container with ID starting with eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085 not found: ID does not exist" containerID="eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085"
Sep 08 15:35:53.995770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.995718    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085"} err="failed to get container status \"eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085\": rpc error: code = NotFound desc = could not find container \"eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085\": container with ID starting with eccee9b61eb54bd2cbea83b822106543badd67281733c4f9adcd4c86a3b81085 not found: ID does not exist"
Sep 08 15:35:53.999850 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:53.999834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.002319 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.002298    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 15:35:54.002405 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.002367    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 15:35:54.002471 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.002452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2kecbjccmih1l\""
Sep 08 15:35:54.002590 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.002561    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 15:35:54.002685 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.002598    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 15:35:54.002991 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.002973    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 15:35:54.003072 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.003061    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 15:35:54.003190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.003175    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 15:35:54.003357 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.003340    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 15:35:54.003889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.003872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 15:35:54.003970 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.003948    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 15:35:54.004033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.004005    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 15:35:54.004364 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.004346    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-djg2d\""
Sep 08 15:35:54.004444 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.004354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 15:35:54.004503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.004438    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 15:35:54.009890 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.009860    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 15:35:54.013150 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.012892    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 15:35:54.016652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.016627    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:35:54.099221 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.099187    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b1ba2e8-ed7e-4834-8f27-cac9836b466c" path="/var/lib/kubelet/pods/9b1ba2e8-ed7e-4834-8f27-cac9836b466c/volumes"
Sep 08 15:35:54.109596 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109763 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109763 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109763 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109763 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109823    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109923    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.109946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nmjl\" (UniqueName: \"kubernetes.io/projected/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-kube-api-access-2nmjl\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.109983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-config\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.110199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.110159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211024 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.210926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211024 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.210969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211024 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.210986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211024 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nmjl\" (UniqueName: \"kubernetes.io/projected/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-kube-api-access-2nmjl\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-config\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.211779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.211560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.212114 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.212045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.212968 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.212546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.213704 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.213681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.214330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.214815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.214866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.215035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.215194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.215206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.215228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215783 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.215610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.215870 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.215840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.216624 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.216181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.216624 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.216587    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.216624 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.216599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.216624 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.216594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-config\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.217586 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.217561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.218344 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.218324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.218509 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.218491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.222269 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.222244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nmjl\" (UniqueName: \"kubernetes.io/projected/7edfe10f-82b1-4f43-b3f7-c6790651dd6d-kube-api-access-2nmjl\") pod \"prometheus-k8s-0\" (UID: \"7edfe10f-82b1-4f43-b3f7-c6790651dd6d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.316513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.316474    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:35:54.602245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.602219    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 15:35:54.604571 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:35:54.604545    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7edfe10f_82b1_4f43_b3f7_c6790651dd6d.slice/crio-982bc4d827c29ad0a73e2e814929adc2d50a73ef9c8173b140dccc9bafea0590 WatchSource:0}: Error finding container 982bc4d827c29ad0a73e2e814929adc2d50a73ef9c8173b140dccc9bafea0590: Status 404 returned error can't find the container with id 982bc4d827c29ad0a73e2e814929adc2d50a73ef9c8173b140dccc9bafea0590
Sep 08 15:35:54.944177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.944145    2563 generic.go:356] "Generic (PLEG): container finished" podID="7edfe10f-82b1-4f43-b3f7-c6790651dd6d" containerID="af7334d1b83be0899f80072997276d03f440eb284f9ccb6918d2fdc228b96820" exitCode=0
Sep 08 15:35:54.944612 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.944228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerDied","Data":"af7334d1b83be0899f80072997276d03f440eb284f9ccb6918d2fdc228b96820"}
Sep 08 15:35:54.944612 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:54.944325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"982bc4d827c29ad0a73e2e814929adc2d50a73ef9c8173b140dccc9bafea0590"}
Sep 08 15:35:55.952396 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.952362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"f3eb3ddd56e3364fa9b52721f7b69f0994c8a60c11831e00928443fd1cb175e0"}
Sep 08 15:35:55.952760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.952467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"2408dd009f7948e7ab80405bb740c3685ac800ed5b03c2e2e4c8f550c79e7596"}
Sep 08 15:35:55.952760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.952496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"b05b5e05c15e09be4943503ee8010c6783bcbbd7d06e64eeeb747ab39363fc7d"}
Sep 08 15:35:55.952760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.952528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"53ceac0796f0020d33d9ba66fe1151bc637fedcaee9af254cc5ef1477cd5d207"}
Sep 08 15:35:55.952760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.952559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"652b6662b2f04cd30790623e7342b0ef164ab523100fdac14d3ad6fb54f4114e"}
Sep 08 15:35:55.952760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.952582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7edfe10f-82b1-4f43-b3f7-c6790651dd6d","Type":"ContainerStarted","Data":"162d28c6fa39780c40fe21be2acf4235a55d13ee46d193aac29fe46115eb4710"}
Sep 08 15:35:55.991524 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:55.991460    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.97459944 podStartE2EDuration="2.991438111s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.838671ms" isStatefulPod=false podCreationTimestamp="2026-09-08 15:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:35:55.987085557 +0000 UTC m=+202.430277796" watchObservedRunningTime="2026-09-08 15:35:55.991438111 +0000 UTC m=+202.434630322"
Sep 08 15:35:59.317264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:35:59.317219    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:36:54.317168 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:36:54.317109    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:36:54.333144 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:36:54.333114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:36:55.210293 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:36:55.210233    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 15:37:34.022043 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:37:34.021967    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 15:38:02.903805 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.903773    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"]
Sep 08 15:38:02.906940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.906924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:02.916222 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.916203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 15:38:02.916469 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.916431    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 15:38:02.916539 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.916467    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 15:38:02.916948 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.916933    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pvhm4\""
Sep 08 15:38:02.916948 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.916941    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 15:38:02.951205 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:02.951181    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"]
Sep 08 15:38:03.039656 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.039615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-manifests\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.039846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.039663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-tmp\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.039846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.039695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.039846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.039744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9qxf\" (UniqueName: \"kubernetes.io/projected/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-kube-api-access-g9qxf\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.039846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.039768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.141045 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.141007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-tmp\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.141206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.141061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.141206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.141123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9qxf\" (UniqueName: \"kubernetes.io/projected/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-kube-api-access-g9qxf\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.141206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.141152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.141206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.141195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-manifests\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.141566 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.141546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-manifests\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.143453 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.143421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-tmp\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.143662 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.143643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-webhook-cert\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.143758 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.143738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.151755 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.151729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9qxf\" (UniqueName: \"kubernetes.io/projected/0a69ef28-d232-4778-aabe-c85b4ca4cf2b-kube-api-access-g9qxf\") pod \"opendatahub-operator-controller-manager-84d887cf8-bk6hl\" (UID: \"0a69ef28-d232-4778-aabe-c85b4ca4cf2b\") " pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.216451 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.216397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:03.516528 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.516500    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"]
Sep 08 15:38:03.518670 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:38:03.518641    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a69ef28_d232_4778_aabe_c85b4ca4cf2b.slice/crio-80137e35845fbc5f8bb56500283b0ce034b511db17a7d0b3b9e58f877ad4aabd WatchSource:0}: Error finding container 80137e35845fbc5f8bb56500283b0ce034b511db17a7d0b3b9e58f877ad4aabd: Status 404 returned error can't find the container with id 80137e35845fbc5f8bb56500283b0ce034b511db17a7d0b3b9e58f877ad4aabd
Sep 08 15:38:03.520496 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:03.520479    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:38:04.475102 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:04.475064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl" event={"ID":"0a69ef28-d232-4778-aabe-c85b4ca4cf2b","Type":"ContainerStarted","Data":"80137e35845fbc5f8bb56500283b0ce034b511db17a7d0b3b9e58f877ad4aabd"}
Sep 08 15:38:06.485864 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:06.485834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl" event={"ID":"0a69ef28-d232-4778-aabe-c85b4ca4cf2b","Type":"ContainerStarted","Data":"58778059cfda882252e1709cdb4467846c54d6465607b30bc09a3510210acfed"}
Sep 08 15:38:08.496497 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:08.496461    2563 generic.go:356] "Generic (PLEG): container finished" podID="0a69ef28-d232-4778-aabe-c85b4ca4cf2b" containerID="58778059cfda882252e1709cdb4467846c54d6465607b30bc09a3510210acfed" exitCode=0
Sep 08 15:38:08.496892 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:08.496547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl" event={"ID":"0a69ef28-d232-4778-aabe-c85b4ca4cf2b","Type":"ContainerDied","Data":"58778059cfda882252e1709cdb4467846c54d6465607b30bc09a3510210acfed"}
Sep 08 15:38:11.512331 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:11.512270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl" event={"ID":"0a69ef28-d232-4778-aabe-c85b4ca4cf2b","Type":"ContainerStarted","Data":"25c28b0f592654ab33d6ff2cc2b8d16d40c3f4b58b38276d8623010336d3935b"}
Sep 08 15:38:11.512728 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:11.512544    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:11.550549 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:11.550491    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl" podStartSLOduration=3.473382429 podStartE2EDuration="9.550472803s" totalImagesPullingTime="5.002800279s" totalInitContainerRuntime="1.074290095s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:38:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:38:11.548492721 +0000 UTC m=+337.991684950" watchObservedRunningTime="2026-09-08 15:38:11.550472803 +0000 UTC m=+337.993665012"
Sep 08 15:38:22.519211 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:22.519180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84d887cf8-bk6hl"
Sep 08 15:38:44.808127 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.808042    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"]
Sep 08 15:38:44.813185 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.813163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.817143 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.817119    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-k2l7p\""
Sep 08 15:38:44.817566 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.817549    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 15:38:44.845962 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.845931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"]
Sep 08 15:38:44.877616 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.877616 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.877887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.877887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.877887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.877887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.877887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.878093 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ldzz\" (UniqueName: \"kubernetes.io/projected/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-kube-api-access-8ldzz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.878093 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.877940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.978725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.978693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.978909 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.978746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.978909 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.978856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ldzz\" (UniqueName: \"kubernetes.io/projected/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-kube-api-access-8ldzz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979029 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.978915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979029 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.978968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979029 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979139    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979603 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979729 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979909 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.979999 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.979756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.981597 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.981568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.982029 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.982009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.996602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.996569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ldzz\" (UniqueName: \"kubernetes.io/projected/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-kube-api-access-8ldzz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:44.996766 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:44.996726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5c9c616-f8dd-4a83-bedd-a2a3ddf86609-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4\" (UID: \"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:45.123776 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:45.123672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:45.434042 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:38:45.434002    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c9c616_f8dd_4a83_bedd_a2a3ddf86609.slice/crio-0ffc388f1c79cef6f721eadc352da16e1e5e797605355bf08db12712ae004fe6 WatchSource:0}: Error finding container 0ffc388f1c79cef6f721eadc352da16e1e5e797605355bf08db12712ae004fe6: Status 404 returned error can't find the container with id 0ffc388f1c79cef6f721eadc352da16e1e5e797605355bf08db12712ae004fe6
Sep 08 15:38:45.434042 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:45.434017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"]
Sep 08 15:38:45.655200 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:45.655160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4" event={"ID":"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609","Type":"ContainerStarted","Data":"0ffc388f1c79cef6f721eadc352da16e1e5e797605355bf08db12712ae004fe6"}
Sep 08 15:38:46.067788 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.067745    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"]
Sep 08 15:38:46.072467 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.072441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.088604    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089029    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2b0d25fa-aaed-4eb1-b326-28ff04552700-manager-config\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b0d25fa-aaed-4eb1-b326-28ff04552700-cert\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blqp8\" (UniqueName: \"kubernetes.io/projected/2b0d25fa-aaed-4eb1-b326-28ff04552700-kube-api-access-blqp8\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089633    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b0d25fa-aaed-4eb1-b326-28ff04552700-metrics-cert\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.090001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.089850    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b8fc8\""
Sep 08 15:38:46.092074 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.092046    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 15:38:46.105114 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.105072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"]
Sep 08 15:38:46.190617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.190583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2b0d25fa-aaed-4eb1-b326-28ff04552700-manager-config\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.190617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.190618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b0d25fa-aaed-4eb1-b326-28ff04552700-cert\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.190870 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.190643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blqp8\" (UniqueName: \"kubernetes.io/projected/2b0d25fa-aaed-4eb1-b326-28ff04552700-kube-api-access-blqp8\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.190870 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.190701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b0d25fa-aaed-4eb1-b326-28ff04552700-metrics-cert\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.191347 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.191320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2b0d25fa-aaed-4eb1-b326-28ff04552700-manager-config\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.193218 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.193192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b0d25fa-aaed-4eb1-b326-28ff04552700-metrics-cert\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.193337 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.193250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b0d25fa-aaed-4eb1-b326-28ff04552700-cert\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.255541 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.255502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blqp8\" (UniqueName: \"kubernetes.io/projected/2b0d25fa-aaed-4eb1-b326-28ff04552700-kube-api-access-blqp8\") pod \"lws-controller-manager-66d9b677c7-r5ttk\" (UID: \"2b0d25fa-aaed-4eb1-b326-28ff04552700\") " pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.382533 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.382440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:46.689573 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:38:46.689536    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b0d25fa_aaed_4eb1_b326_28ff04552700.slice/crio-8276fd6ae980015d3548311d947eaf94d2646c06045d5b42731d2faaea0825ed WatchSource:0}: Error finding container 8276fd6ae980015d3548311d947eaf94d2646c06045d5b42731d2faaea0825ed: Status 404 returned error can't find the container with id 8276fd6ae980015d3548311d947eaf94d2646c06045d5b42731d2faaea0825ed
Sep 08 15:38:46.693762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:46.693735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"]
Sep 08 15:38:47.667358 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:47.667313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk" event={"ID":"2b0d25fa-aaed-4eb1-b326-28ff04552700","Type":"ContainerStarted","Data":"8276fd6ae980015d3548311d947eaf94d2646c06045d5b42731d2faaea0825ed"}
Sep 08 15:38:48.171832 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.171794    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"]
Sep 08 15:38:48.175429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.175401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.212903 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.212839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq7dh\" (UniqueName: \"kubernetes.io/projected/6a31c7af-5184-42fe-a823-7aaae4c2a2ec-kube-api-access-zq7dh\") pod \"kserve-controller-manager-d6f4b65b8-bll9l\" (UID: \"6a31c7af-5184-42fe-a823-7aaae4c2a2ec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.213081 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.212930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a31c7af-5184-42fe-a823-7aaae4c2a2ec-cert\") pod \"kserve-controller-manager-d6f4b65b8-bll9l\" (UID: \"6a31c7af-5184-42fe-a823-7aaae4c2a2ec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.287887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.287662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 08 15:38:48.287887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.287663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2fq6b\""
Sep 08 15:38:48.289673 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.289644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"]
Sep 08 15:38:48.318940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.318335    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq7dh\" (UniqueName: \"kubernetes.io/projected/6a31c7af-5184-42fe-a823-7aaae4c2a2ec-kube-api-access-zq7dh\") pod \"kserve-controller-manager-d6f4b65b8-bll9l\" (UID: \"6a31c7af-5184-42fe-a823-7aaae4c2a2ec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.318940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.318417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a31c7af-5184-42fe-a823-7aaae4c2a2ec-cert\") pod \"kserve-controller-manager-d6f4b65b8-bll9l\" (UID: \"6a31c7af-5184-42fe-a823-7aaae4c2a2ec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.348310 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.347664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a31c7af-5184-42fe-a823-7aaae4c2a2ec-cert\") pod \"kserve-controller-manager-d6f4b65b8-bll9l\" (UID: \"6a31c7af-5184-42fe-a823-7aaae4c2a2ec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.348310 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.348266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq7dh\" (UniqueName: \"kubernetes.io/projected/6a31c7af-5184-42fe-a823-7aaae4c2a2ec-kube-api-access-zq7dh\") pod \"kserve-controller-manager-d6f4b65b8-bll9l\" (UID: \"6a31c7af-5184-42fe-a823-7aaae4c2a2ec\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:48.489365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:48.489327    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:49.880945 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:49.880900    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:38:49.881334 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:49.880996    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:38:49.881334 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:49.881047    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:38:50.166826 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.166607    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"]
Sep 08 15:38:50.169639 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:38:50.169606    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a31c7af_5184_42fe_a823_7aaae4c2a2ec.slice/crio-a74dfbe79fd1f43371f77d53faf9661fafea5ec3a2a4b9a4d00107cc00440926 WatchSource:0}: Error finding container a74dfbe79fd1f43371f77d53faf9661fafea5ec3a2a4b9a4d00107cc00440926: Status 404 returned error can't find the container with id a74dfbe79fd1f43371f77d53faf9661fafea5ec3a2a4b9a4d00107cc00440926
Sep 08 15:38:50.683106 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.683072    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4" event={"ID":"d5c9c616-f8dd-4a83-bedd-a2a3ddf86609","Type":"ContainerStarted","Data":"a5fad8b1b6576de4c6f01c837a51c48f320a2934c232b306f6be89a9282c7f41"}
Sep 08 15:38:50.684558 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.684531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk" event={"ID":"2b0d25fa-aaed-4eb1-b326-28ff04552700","Type":"ContainerStarted","Data":"2e9606e86e17cfff9c1e83f938edbb95af5d4bf5f325bb03d45ea41bffa70fae"}
Sep 08 15:38:50.684693 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.684586    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:38:50.685723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.685702    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l" event={"ID":"6a31c7af-5184-42fe-a823-7aaae4c2a2ec","Type":"ContainerStarted","Data":"a74dfbe79fd1f43371f77d53faf9661fafea5ec3a2a4b9a4d00107cc00440926"}
Sep 08 15:38:50.744975 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.744926    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4" podStartSLOduration=2.301172958 podStartE2EDuration="6.744909257s" totalImagesPullingTime="4.443736299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:38:50.743032876 +0000 UTC m=+377.186225084" watchObservedRunningTime="2026-09-08 15:38:50.744909257 +0000 UTC m=+377.188101465"
Sep 08 15:38:50.782087 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:50.782026    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk" podStartSLOduration=1.537844948 podStartE2EDuration="4.782008147s" totalImagesPullingTime="3.244163199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:38:50.781153398 +0000 UTC m=+377.224345628" watchObservedRunningTime="2026-09-08 15:38:50.782008147 +0000 UTC m=+377.225200356"
Sep 08 15:38:51.124877 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:51.124739    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:51.130756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:51.130693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:51.690598 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:51.690558    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:51.691641 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:51.691616    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4"
Sep 08 15:38:53.701661 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:53.701614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l" event={"ID":"6a31c7af-5184-42fe-a823-7aaae4c2a2ec","Type":"ContainerStarted","Data":"90f6e6aab30dcb0e4002432f48188462ebae2eb705a3ae5eef51e4a5683de154"}
Sep 08 15:38:53.702044 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:53.701849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:38:53.732005 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:38:53.731949    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l" podStartSLOduration=3.055908941 podStartE2EDuration="5.73193409s" totalImagesPullingTime="2.676025149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:38:53.730397173 +0000 UTC m=+380.173589401" watchObservedRunningTime="2026-09-08 15:38:53.73193409 +0000 UTC m=+380.175126298"
Sep 08 15:39:01.692924 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:39:01.692894    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66d9b677c7-r5ttk"
Sep 08 15:39:24.711304 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:39:24.711259    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bll9l"
Sep 08 15:40:22.730963 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.730924    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"]
Sep 08 15:40:22.733348 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.733329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.739011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.738982    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 15:40:22.739011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.738999    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 15:40:22.739216 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.739157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pmd7w\""
Sep 08 15:40:22.757466 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.757430    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"]
Sep 08 15:40:22.762126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.762100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/547ce61e-e904-4799-88d1-ecef96bd78c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-s6zsv\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.762255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.762144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqlvx\" (UniqueName: \"kubernetes.io/projected/547ce61e-e904-4799-88d1-ecef96bd78c0-kube-api-access-mqlvx\") pod \"kuadrant-operator-controller-manager-6bd956ccf-s6zsv\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.837632 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.837595    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"]
Sep 08 15:40:22.837920 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:40:22.837900    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-mqlvx], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv" podUID="547ce61e-e904-4799-88d1-ecef96bd78c0"
Sep 08 15:40:22.862831 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.862798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/547ce61e-e904-4799-88d1-ecef96bd78c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-s6zsv\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.862955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.862849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqlvx\" (UniqueName: \"kubernetes.io/projected/547ce61e-e904-4799-88d1-ecef96bd78c0-kube-api-access-mqlvx\") pod \"kuadrant-operator-controller-manager-6bd956ccf-s6zsv\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.863225 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.863204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/547ce61e-e904-4799-88d1-ecef96bd78c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-s6zsv\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.881994 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.881964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqlvx\" (UniqueName: \"kubernetes.io/projected/547ce61e-e904-4799-88d1-ecef96bd78c0-kube-api-access-mqlvx\") pod \"kuadrant-operator-controller-manager-6bd956ccf-s6zsv\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:22.885271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.885241    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"]
Sep 08 15:40:22.893150 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.893108    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:22.906169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.906138    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"]
Sep 08 15:40:22.963452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.963412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r692h\" (UniqueName: \"kubernetes.io/projected/064c54e8-174c-4fdc-a39b-a6ff2e003647-kube-api-access-r692h\") pod \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:22.963628 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:22.963535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/064c54e8-174c-4fdc-a39b-a6ff2e003647-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.064411 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.064282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/064c54e8-174c-4fdc-a39b-a6ff2e003647-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.064411 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.064393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r692h\" (UniqueName: \"kubernetes.io/projected/064c54e8-174c-4fdc-a39b-a6ff2e003647-kube-api-access-r692h\") pod \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.064756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.064731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/064c54e8-174c-4fdc-a39b-a6ff2e003647-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.073870 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.073837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r692h\" (UniqueName: \"kubernetes.io/projected/064c54e8-174c-4fdc-a39b-a6ff2e003647-kube-api-access-r692h\") pod \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.101400 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.101367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:23.107432 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.107408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:23.164805 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.164765    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqlvx\" (UniqueName: \"kubernetes.io/projected/547ce61e-e904-4799-88d1-ecef96bd78c0-kube-api-access-mqlvx\") pod \"547ce61e-e904-4799-88d1-ecef96bd78c0\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") "
Sep 08 15:40:23.165013 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.164833    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/547ce61e-e904-4799-88d1-ecef96bd78c0-extensions-socket-volume\") pod \"547ce61e-e904-4799-88d1-ecef96bd78c0\" (UID: \"547ce61e-e904-4799-88d1-ecef96bd78c0\") "
Sep 08 15:40:23.165169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.165136    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/547ce61e-e904-4799-88d1-ecef96bd78c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "547ce61e-e904-4799-88d1-ecef96bd78c0" (UID: "547ce61e-e904-4799-88d1-ecef96bd78c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:23.166910 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.166886    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547ce61e-e904-4799-88d1-ecef96bd78c0-kube-api-access-mqlvx" (OuterVolumeSpecName: "kube-api-access-mqlvx") pod "547ce61e-e904-4799-88d1-ecef96bd78c0" (UID: "547ce61e-e904-4799-88d1-ecef96bd78c0"). InnerVolumeSpecName "kube-api-access-mqlvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:40:23.203111 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.203079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.266058 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.265893    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqlvx\" (UniqueName: \"kubernetes.io/projected/547ce61e-e904-4799-88d1-ecef96bd78c0-kube-api-access-mqlvx\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:40:23.266058 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.265934    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/547ce61e-e904-4799-88d1-ecef96bd78c0-extensions-socket-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:40:23.320105 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.320020    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"]
Sep 08 15:40:23.335185 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.335150    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"]
Sep 08 15:40:23.349915 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.349882    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"]
Sep 08 15:40:23.364986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.364945    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"]
Sep 08 15:40:23.404065 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.404034    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"]
Sep 08 15:40:23.406529 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.406511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:23.409388 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.409363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-v7pb9\""
Sep 08 15:40:23.419728 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.419697    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"]
Sep 08 15:40:23.467088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.467043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkd2c\" (UniqueName: \"kubernetes.io/projected/2c95295b-1693-4305-ac64-5b55206b1b2b-kube-api-access-wkd2c\") pod \"limitador-operator-controller-manager-6c549b6d85-wmfqn\" (UID: \"2c95295b-1693-4305-ac64-5b55206b1b2b\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:23.568179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.568139    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkd2c\" (UniqueName: \"kubernetes.io/projected/2c95295b-1693-4305-ac64-5b55206b1b2b-kube-api-access-wkd2c\") pod \"limitador-operator-controller-manager-6c549b6d85-wmfqn\" (UID: \"2c95295b-1693-4305-ac64-5b55206b1b2b\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:23.589459 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.589428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkd2c\" (UniqueName: \"kubernetes.io/projected/2c95295b-1693-4305-ac64-5b55206b1b2b-kube-api-access-wkd2c\") pod \"limitador-operator-controller-manager-6c549b6d85-wmfqn\" (UID: \"2c95295b-1693-4305-ac64-5b55206b1b2b\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:40:23.594308    2563 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-68cf589589-x8xqt_kuadrant-system_064c54e8-174c-4fdc-a39b-a6ff2e003647_0(c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-68cf589589-x8xqt to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9" Netns:"/var/run/netns/a7eb204d-b3f0-47e9-8670-d5a5c21c832a" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-68cf589589-x8xqt;K8S_POD_INFRA_CONTAINER_ID=c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9;K8S_POD_UID=064c54e8-174c-4fdc-a39b-a6ff2e003647" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt] networking: [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt/064c54e8-174c-4fdc-a39b-a6ff2e003647:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:1c [10.133.0.28/23]
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:         '
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:  >
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:40:23.594386    2563 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-68cf589589-x8xqt_kuadrant-system_064c54e8-174c-4fdc-a39b-a6ff2e003647_0(c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-68cf589589-x8xqt to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9" Netns:"/var/run/netns/a7eb204d-b3f0-47e9-8670-d5a5c21c832a" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-68cf589589-x8xqt;K8S_POD_INFRA_CONTAINER_ID=c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9;K8S_POD_UID=064c54e8-174c-4fdc-a39b-a6ff2e003647" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt] networking: [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt/064c54e8-174c-4fdc-a39b-a6ff2e003647:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt c885595adb902785051be5ff5e98121ac4edd66182dd3c691c84bfd83a8543c9 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:1c [10.133.0.28/23]
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:         '
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 08 15:40:23.594537 ip-10-0-135-240 kubenswrapper[2563]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:23.717368 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.717322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:23.995322 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:23.995279    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"]
Sep 08 15:40:23.997504 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:40:23.997477    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c95295b_1693_4305_ac64_5b55206b1b2b.slice/crio-1ba78cee444752623271647bef2da0eab6439b6b8ace9595ebd0a4ddb0f97814 WatchSource:0}: Error finding container 1ba78cee444752623271647bef2da0eab6439b6b8ace9595ebd0a4ddb0f97814: Status 404 returned error can't find the container with id 1ba78cee444752623271647bef2da0eab6439b6b8ace9595ebd0a4ddb0f97814
Sep 08 15:40:24.100325 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.100272    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="547ce61e-e904-4799-88d1-ecef96bd78c0" path="/var/lib/kubelet/pods/547ce61e-e904-4799-88d1-ecef96bd78c0/volumes"
Sep 08 15:40:24.106806 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.106772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn" event={"ID":"2c95295b-1693-4305-ac64-5b55206b1b2b","Type":"ContainerStarted","Data":"1ba78cee444752623271647bef2da0eab6439b6b8ace9595ebd0a4ddb0f97814"}
Sep 08 15:40:24.106947 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.106829    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:24.106947 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.106829    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-s6zsv"
Sep 08 15:40:24.113881 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.113860    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:24.116216 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:40:24.116191    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" is forbidden: User \"system:node:ip-10-0-135-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-240.ec2.internal' and this object" podUID="064c54e8-174c-4fdc-a39b-a6ff2e003647" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:24.174019 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.173992    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r692h\" (UniqueName: \"kubernetes.io/projected/064c54e8-174c-4fdc-a39b-a6ff2e003647-kube-api-access-r692h\") pod \"064c54e8-174c-4fdc-a39b-a6ff2e003647\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") "
Sep 08 15:40:24.174177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.174056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/064c54e8-174c-4fdc-a39b-a6ff2e003647-extensions-socket-volume\") pod \"064c54e8-174c-4fdc-a39b-a6ff2e003647\" (UID: \"064c54e8-174c-4fdc-a39b-a6ff2e003647\") "
Sep 08 15:40:24.174365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.174338    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/064c54e8-174c-4fdc-a39b-a6ff2e003647-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "064c54e8-174c-4fdc-a39b-a6ff2e003647" (UID: "064c54e8-174c-4fdc-a39b-a6ff2e003647"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:40:24.176074 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.176052    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/064c54e8-174c-4fdc-a39b-a6ff2e003647-kube-api-access-r692h" (OuterVolumeSpecName: "kube-api-access-r692h") pod "064c54e8-174c-4fdc-a39b-a6ff2e003647" (UID: "064c54e8-174c-4fdc-a39b-a6ff2e003647"). InnerVolumeSpecName "kube-api-access-r692h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:40:24.274946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.274907    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/064c54e8-174c-4fdc-a39b-a6ff2e003647-extensions-socket-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:40:24.274946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:24.274940    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r692h\" (UniqueName: \"kubernetes.io/projected/064c54e8-174c-4fdc-a39b-a6ff2e003647-kube-api-access-r692h\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:40:25.111861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:25.111828    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:25.114577 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:40:25.114524    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" is forbidden: User \"system:node:ip-10-0-135-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-240.ec2.internal' and this object" podUID="064c54e8-174c-4fdc-a39b-a6ff2e003647" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:25.121944 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:40:25.121905    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-x8xqt\" is forbidden: User \"system:node:ip-10-0-135-240.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-240.ec2.internal' and this object" podUID="064c54e8-174c-4fdc-a39b-a6ff2e003647" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-x8xqt"
Sep 08 15:40:26.101181 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:26.101147    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="064c54e8-174c-4fdc-a39b-a6ff2e003647" path="/var/lib/kubelet/pods/064c54e8-174c-4fdc-a39b-a6ff2e003647/volumes"
Sep 08 15:40:27.123098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:27.123056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn" event={"ID":"2c95295b-1693-4305-ac64-5b55206b1b2b","Type":"ContainerStarted","Data":"3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f"}
Sep 08 15:40:27.123519 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:27.123165    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:27.145006 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:27.144944    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn" podStartSLOduration=1.789686159 podStartE2EDuration="4.144927341s" totalImagesPullingTime="2.355241182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:27.14349775 +0000 UTC m=+473.586689983" watchObservedRunningTime="2026-09-08 15:40:27.144927341 +0000 UTC m=+473.588119550"
Sep 08 15:40:38.130190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:38.130156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:40:52.602759 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.602718    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"]
Sep 08 15:40:52.605940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.605915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.608432 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.608401    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wswb9\""
Sep 08 15:40:52.616786 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.616755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"]
Sep 08 15:40:52.736217 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lppsp\" (UniqueName: \"kubernetes.io/projected/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-kube-api-access-lppsp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736418    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736625 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736625 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736625 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.736625 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.736597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.837648 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.837603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.837859 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.837679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.837859 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.837709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.837859 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.837737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.837859 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.837789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838059 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.837972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lppsp\" (UniqueName: \"kubernetes.io/projected/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-kube-api-access-lppsp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838059 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838161 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838161 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838265 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838571 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838571 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838713 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.838713 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.838636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.840163 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.840138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.840544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.840477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.846726 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.846691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.847062 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.847039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lppsp\" (UniqueName: \"kubernetes.io/projected/18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca-kube-api-access-lppsp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-xprr6\" (UID: \"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:52.917730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:52.917628    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:53.196962 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:53.196925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"]
Sep 08 15:40:53.199767 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:40:53.199732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18cb8a11_7ae1_4a8b_a2d3_0e0b084b5bca.slice/crio-7e4de5f196e81727ce30b708ef94bd4227a8ec3485f0834f467fc8f8be3d0c7e WatchSource:0}: Error finding container 7e4de5f196e81727ce30b708ef94bd4227a8ec3485f0834f467fc8f8be3d0c7e: Status 404 returned error can't find the container with id 7e4de5f196e81727ce30b708ef94bd4227a8ec3485f0834f467fc8f8be3d0c7e
Sep 08 15:40:53.201917 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:53.201882    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:40:53.202012 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:53.201970    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:40:53.202074 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:53.202013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:40:53.247172 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:53.247143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" event={"ID":"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca","Type":"ContainerStarted","Data":"7e4de5f196e81727ce30b708ef94bd4227a8ec3485f0834f467fc8f8be3d0c7e"}
Sep 08 15:40:54.253508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:54.253471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" event={"ID":"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca","Type":"ContainerStarted","Data":"60ba7b504c30972b3b6beb7027771ac92d0a8e4aa297e98c7db6d69f196f7a7a"}
Sep 08 15:40:54.273744 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:54.273689    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" podStartSLOduration=2.273666623 podStartE2EDuration="2.273666623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:40:54.271706857 +0000 UTC m=+500.714899077" watchObservedRunningTime="2026-09-08 15:40:54.273666623 +0000 UTC m=+500.716858832"
Sep 08 15:40:54.917867 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:54.917830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:54.923397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:54.923360    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:55.257949 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:55.257918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:40:55.258887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:40:55.258867    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:41:12.056553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.056521    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-tpnsr"]
Sep 08 15:41:12.060730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.060710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:12.063278 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.063257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2r4ms\""
Sep 08 15:41:12.068123 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.067812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-tpnsr"]
Sep 08 15:41:12.214068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.214022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjwxp\" (UniqueName: \"kubernetes.io/projected/6e1801a9-80ee-4c47-b245-4bb110b465e3-kube-api-access-vjwxp\") pod \"authorino-b54df94b4-tpnsr\" (UID: \"6e1801a9-80ee-4c47-b245-4bb110b465e3\") " pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:12.315334 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.315210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjwxp\" (UniqueName: \"kubernetes.io/projected/6e1801a9-80ee-4c47-b245-4bb110b465e3-kube-api-access-vjwxp\") pod \"authorino-b54df94b4-tpnsr\" (UID: \"6e1801a9-80ee-4c47-b245-4bb110b465e3\") " pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:12.323629 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.323598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjwxp\" (UniqueName: \"kubernetes.io/projected/6e1801a9-80ee-4c47-b245-4bb110b465e3-kube-api-access-vjwxp\") pod \"authorino-b54df94b4-tpnsr\" (UID: \"6e1801a9-80ee-4c47-b245-4bb110b465e3\") " pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:12.370687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.370639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:12.640702 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:12.640671    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-tpnsr"]
Sep 08 15:41:12.643194 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:41:12.643168    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e1801a9_80ee_4c47_b245_4bb110b465e3.slice/crio-d422168dc8ec6c45f11e0031a18968fbaf69ca63ef5bab881db70fa777b44434 WatchSource:0}: Error finding container d422168dc8ec6c45f11e0031a18968fbaf69ca63ef5bab881db70fa777b44434: Status 404 returned error can't find the container with id d422168dc8ec6c45f11e0031a18968fbaf69ca63ef5bab881db70fa777b44434
Sep 08 15:41:13.344099 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:13.344061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-tpnsr" event={"ID":"6e1801a9-80ee-4c47-b245-4bb110b465e3","Type":"ContainerStarted","Data":"d422168dc8ec6c45f11e0031a18968fbaf69ca63ef5bab881db70fa777b44434"}
Sep 08 15:41:15.497650 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:15.497614    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-tpnsr"]
Sep 08 15:41:17.365007 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.364969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-tpnsr" event={"ID":"6e1801a9-80ee-4c47-b245-4bb110b465e3","Type":"ContainerStarted","Data":"b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34"}
Sep 08 15:41:17.365007 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.364981    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-tpnsr" podUID="6e1801a9-80ee-4c47-b245-4bb110b465e3" containerName="authorino" containerID="cri-o://b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34" gracePeriod=30
Sep 08 15:41:17.383251 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.383203    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-tpnsr" podStartSLOduration=1.173405163 podStartE2EDuration="5.383183815s" totalImagesPullingTime="4.209778652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:17.381383938 +0000 UTC m=+523.824576146" watchObservedRunningTime="2026-09-08 15:41:17.383183815 +0000 UTC m=+523.826376023"
Sep 08 15:41:17.608300 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.608263    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:17.661523 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.661434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjwxp\" (UniqueName: \"kubernetes.io/projected/6e1801a9-80ee-4c47-b245-4bb110b465e3-kube-api-access-vjwxp\") pod \"6e1801a9-80ee-4c47-b245-4bb110b465e3\" (UID: \"6e1801a9-80ee-4c47-b245-4bb110b465e3\") "
Sep 08 15:41:17.663679 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.663648    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1801a9-80ee-4c47-b245-4bb110b465e3-kube-api-access-vjwxp" (OuterVolumeSpecName: "kube-api-access-vjwxp") pod "6e1801a9-80ee-4c47-b245-4bb110b465e3" (UID: "6e1801a9-80ee-4c47-b245-4bb110b465e3"). InnerVolumeSpecName "kube-api-access-vjwxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:17.762099 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:17.762053    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjwxp\" (UniqueName: \"kubernetes.io/projected/6e1801a9-80ee-4c47-b245-4bb110b465e3-kube-api-access-vjwxp\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:41:18.370890 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.370805    2563 generic.go:356] "Generic (PLEG): container finished" podID="6e1801a9-80ee-4c47-b245-4bb110b465e3" containerID="b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34" exitCode=0
Sep 08 15:41:18.370890 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.370843    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-tpnsr"
Sep 08 15:41:18.371403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.370888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-tpnsr" event={"ID":"6e1801a9-80ee-4c47-b245-4bb110b465e3","Type":"ContainerDied","Data":"b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34"}
Sep 08 15:41:18.371403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.370927    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-tpnsr" event={"ID":"6e1801a9-80ee-4c47-b245-4bb110b465e3","Type":"ContainerDied","Data":"d422168dc8ec6c45f11e0031a18968fbaf69ca63ef5bab881db70fa777b44434"}
Sep 08 15:41:18.371403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.370945    2563 scope.go:117] "RemoveContainer" containerID="b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34"
Sep 08 15:41:18.381255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.381228    2563 scope.go:117] "RemoveContainer" containerID="b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34"
Sep 08 15:41:18.381861 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:41:18.381839    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34\": container with ID starting with b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34 not found: ID does not exist" containerID="b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34"
Sep 08 15:41:18.381951 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.381869    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34"} err="failed to get container status \"b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34\": rpc error: code = NotFound desc = could not find container \"b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34\": container with ID starting with b7c192c0bdee40c450c981e60936edbc741dc0e249227b75d43d640dc16a6d34 not found: ID does not exist"
Sep 08 15:41:18.388188 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.388167    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-tpnsr"]
Sep 08 15:41:18.391850 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:18.391828    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-tpnsr"]
Sep 08 15:41:20.100676 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:20.100647    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1801a9-80ee-4c47-b245-4bb110b465e3" path="/var/lib/kubelet/pods/6e1801a9-80ee-4c47-b245-4bb110b465e3/volumes"
Sep 08 15:41:27.229925 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.229842    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bf726"]
Sep 08 15:41:27.230403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.230213    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1801a9-80ee-4c47-b245-4bb110b465e3" containerName="authorino"
Sep 08 15:41:27.230403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.230225    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1801a9-80ee-4c47-b245-4bb110b465e3" containerName="authorino"
Sep 08 15:41:27.230403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.230315    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1801a9-80ee-4c47-b245-4bb110b465e3" containerName="authorino"
Sep 08 15:41:27.236898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.236857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.241123 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.241093    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 15:41:27.241537 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.241511    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 15:41:27.242014 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.241991    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 15:41:27.242116 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.242020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-2npqh\""
Sep 08 15:41:27.247557 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.247530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bf726"]
Sep 08 15:41:27.343939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.343777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dd06e090-eda2-4ddc-9149-5f5920b07847-data\") pod \"postgres-69fb4594cf-bf726\" (UID: \"dd06e090-eda2-4ddc-9149-5f5920b07847\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.343939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.343825    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-479wd\" (UniqueName: \"kubernetes.io/projected/dd06e090-eda2-4ddc-9149-5f5920b07847-kube-api-access-479wd\") pod \"postgres-69fb4594cf-bf726\" (UID: \"dd06e090-eda2-4ddc-9149-5f5920b07847\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.445276 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.445237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dd06e090-eda2-4ddc-9149-5f5920b07847-data\") pod \"postgres-69fb4594cf-bf726\" (UID: \"dd06e090-eda2-4ddc-9149-5f5920b07847\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.445276 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.445278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-479wd\" (UniqueName: \"kubernetes.io/projected/dd06e090-eda2-4ddc-9149-5f5920b07847-kube-api-access-479wd\") pod \"postgres-69fb4594cf-bf726\" (UID: \"dd06e090-eda2-4ddc-9149-5f5920b07847\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.445688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.445659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dd06e090-eda2-4ddc-9149-5f5920b07847-data\") pod \"postgres-69fb4594cf-bf726\" (UID: \"dd06e090-eda2-4ddc-9149-5f5920b07847\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.453550 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.453519    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-479wd\" (UniqueName: \"kubernetes.io/projected/dd06e090-eda2-4ddc-9149-5f5920b07847-kube-api-access-479wd\") pod \"postgres-69fb4594cf-bf726\" (UID: \"dd06e090-eda2-4ddc-9149-5f5920b07847\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.551406 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.551313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:27.827279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:27.827127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-bf726"]
Sep 08 15:41:27.830309 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:41:27.830252    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd06e090_eda2_4ddc_9149_5f5920b07847.slice/crio-38b51f6775dcb5097a83e1a415a4d68782a6eaa04784cfdcfdda0a398e2f430a WatchSource:0}: Error finding container 38b51f6775dcb5097a83e1a415a4d68782a6eaa04784cfdcfdda0a398e2f430a: Status 404 returned error can't find the container with id 38b51f6775dcb5097a83e1a415a4d68782a6eaa04784cfdcfdda0a398e2f430a
Sep 08 15:41:28.418753 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:28.418707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726" event={"ID":"dd06e090-eda2-4ddc-9149-5f5920b07847","Type":"ContainerStarted","Data":"38b51f6775dcb5097a83e1a415a4d68782a6eaa04784cfdcfdda0a398e2f430a"}
Sep 08 15:41:33.446833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:33.446786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726" event={"ID":"dd06e090-eda2-4ddc-9149-5f5920b07847","Type":"ContainerStarted","Data":"52cfda056e328e7b78915047317fbe9accce2fe5567fdfd5b4b7927ab9e75cb7"}
Sep 08 15:41:33.447300 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:33.446849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:33.462534 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:33.462479    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726" podStartSLOduration=1.741550235 podStartE2EDuration="6.462461283s" totalImagesPullingTime="4.720911048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:33.461133926 +0000 UTC m=+539.904326154" watchObservedRunningTime="2026-09-08 15:41:33.462461283 +0000 UTC m=+539.905653492"
Sep 08 15:41:39.481371 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:39.481340    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-bf726"
Sep 08 15:41:40.102705 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.102670    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-z59th"]
Sep 08 15:41:40.108064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.108041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.111251 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.111230    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2r4ms\""
Sep 08 15:41:40.111385 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.111257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 08 15:41:40.114548 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.114526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-z59th"]
Sep 08 15:41:40.168265 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.168230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9475f404-763d-45d0-b98a-b2435c40a571-tls-cert\") pod \"authorino-687bd45585-z59th\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") " pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.168265 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.168266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kn4h\" (UniqueName: \"kubernetes.io/projected/9475f404-763d-45d0-b98a-b2435c40a571-kube-api-access-2kn4h\") pod \"authorino-687bd45585-z59th\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") " pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.268928 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.268894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9475f404-763d-45d0-b98a-b2435c40a571-tls-cert\") pod \"authorino-687bd45585-z59th\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") " pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.268928 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.268927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kn4h\" (UniqueName: \"kubernetes.io/projected/9475f404-763d-45d0-b98a-b2435c40a571-kube-api-access-2kn4h\") pod \"authorino-687bd45585-z59th\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") " pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.271268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.271244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9475f404-763d-45d0-b98a-b2435c40a571-tls-cert\") pod \"authorino-687bd45585-z59th\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") " pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.273659 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.273635    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-z59th"]
Sep 08 15:41:40.273915 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:41:40.273897    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2kn4h], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-z59th" podUID="9475f404-763d-45d0-b98a-b2435c40a571"
Sep 08 15:41:40.276831 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.276811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kn4h\" (UniqueName: \"kubernetes.io/projected/9475f404-763d-45d0-b98a-b2435c40a571-kube-api-access-2kn4h\") pod \"authorino-687bd45585-z59th\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") " pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.479728 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.479695    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.487088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.487063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:40.571515 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.571478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kn4h\" (UniqueName: \"kubernetes.io/projected/9475f404-763d-45d0-b98a-b2435c40a571-kube-api-access-2kn4h\") pod \"9475f404-763d-45d0-b98a-b2435c40a571\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") "
Sep 08 15:41:40.571692 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.571574    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9475f404-763d-45d0-b98a-b2435c40a571-tls-cert\") pod \"9475f404-763d-45d0-b98a-b2435c40a571\" (UID: \"9475f404-763d-45d0-b98a-b2435c40a571\") "
Sep 08 15:41:40.573598 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.573575    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9475f404-763d-45d0-b98a-b2435c40a571-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9475f404-763d-45d0-b98a-b2435c40a571" (UID: "9475f404-763d-45d0-b98a-b2435c40a571"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:41:40.573598 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.573586    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9475f404-763d-45d0-b98a-b2435c40a571-kube-api-access-2kn4h" (OuterVolumeSpecName: "kube-api-access-2kn4h") pod "9475f404-763d-45d0-b98a-b2435c40a571" (UID: "9475f404-763d-45d0-b98a-b2435c40a571"). InnerVolumeSpecName "kube-api-access-2kn4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:41:40.672641 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.672596    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kn4h\" (UniqueName: \"kubernetes.io/projected/9475f404-763d-45d0-b98a-b2435c40a571-kube-api-access-2kn4h\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:41:40.672641 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:40.672633    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9475f404-763d-45d0-b98a-b2435c40a571-tls-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:41:41.484199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:41.484165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-z59th"
Sep 08 15:41:41.513902 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:41.513874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-z59th"]
Sep 08 15:41:41.517225 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:41.517193    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-z59th"]
Sep 08 15:41:42.100722 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:42.100686    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9475f404-763d-45d0-b98a-b2435c40a571" path="/var/lib/kubelet/pods/9475f404-763d-45d0-b98a-b2435c40a571/volumes"
Sep 08 15:41:46.412219 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.412180    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c598fcdf9-2x9sv"]
Sep 08 15:41:46.430703 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.430666    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c598fcdf9-2x9sv"]
Sep 08 15:41:46.430881 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.430787    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.433733 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.433709    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qvnl7\""
Sep 08 15:41:46.433919 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.433898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 15:41:46.434416 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.434392    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 15:41:46.435096 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.435077    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 15:41:46.531254 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.531214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-metrics-certs\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.531468 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.531332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq8c6\" (UniqueName: \"kubernetes.io/projected/1ba9b772-78c2-4a69-9489-36e0a51b6404-kube-api-access-dq8c6\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.531468 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.531374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-webhook-cert\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.632710 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.632668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq8c6\" (UniqueName: \"kubernetes.io/projected/1ba9b772-78c2-4a69-9489-36e0a51b6404-kube-api-access-dq8c6\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.632881 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.632733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-webhook-cert\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.632881 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.632776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-metrics-certs\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.635316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.635277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-metrics-certs\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.635316 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.635307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-webhook-cert\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.641440 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.641409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq8c6\" (UniqueName: \"kubernetes.io/projected/1ba9b772-78c2-4a69-9489-36e0a51b6404-kube-api-access-dq8c6\") pod \"maas-controller-6c598fcdf9-2x9sv\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") " pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:46.746754 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:46.746710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:47.020422 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:47.020368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c598fcdf9-2x9sv"]
Sep 08 15:41:47.022977 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:41:47.022947    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ba9b772_78c2_4a69_9489_36e0a51b6404.slice/crio-b36f0d2a08dd20db0fe602edaf1de9e6c08ea4ddaca18cab27e0914f645899c3 WatchSource:0}: Error finding container b36f0d2a08dd20db0fe602edaf1de9e6c08ea4ddaca18cab27e0914f645899c3: Status 404 returned error can't find the container with id b36f0d2a08dd20db0fe602edaf1de9e6c08ea4ddaca18cab27e0914f645899c3
Sep 08 15:41:47.515654 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:47.515609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv" event={"ID":"1ba9b772-78c2-4a69-9489-36e0a51b6404","Type":"ContainerStarted","Data":"b36f0d2a08dd20db0fe602edaf1de9e6c08ea4ddaca18cab27e0914f645899c3"}
Sep 08 15:41:50.534647 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:50.534610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv" event={"ID":"1ba9b772-78c2-4a69-9489-36e0a51b6404","Type":"ContainerStarted","Data":"ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9"}
Sep 08 15:41:50.535055 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:50.534725    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:41:50.553676 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:41:50.553622    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv" podStartSLOduration=2.013632509 podStartE2EDuration="4.55360502s" totalImagesPullingTime="2.539972511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:41:50.552193445 +0000 UTC m=+556.995385674" watchObservedRunningTime="2026-09-08 15:41:50.55360502 +0000 UTC m=+556.996797230"
Sep 08 15:42:01.541962 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:01.541930    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:42:04.720958 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:04.720914    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c598fcdf9-2x9sv"]
Sep 08 15:42:04.721398 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:04.721167    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv" podUID="1ba9b772-78c2-4a69-9489-36e0a51b6404" containerName="manager" containerID="cri-o://ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9" gracePeriod=30
Sep 08 15:42:04.966600 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:04.966571    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:42:05.003811 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.003728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-webhook-cert\") pod \"1ba9b772-78c2-4a69-9489-36e0a51b6404\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") "
Sep 08 15:42:05.003811 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.003785    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq8c6\" (UniqueName: \"kubernetes.io/projected/1ba9b772-78c2-4a69-9489-36e0a51b6404-kube-api-access-dq8c6\") pod \"1ba9b772-78c2-4a69-9489-36e0a51b6404\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") "
Sep 08 15:42:05.004022 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.003835    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-metrics-certs\") pod \"1ba9b772-78c2-4a69-9489-36e0a51b6404\" (UID: \"1ba9b772-78c2-4a69-9489-36e0a51b6404\") "
Sep 08 15:42:05.006054 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.006016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1ba9b772-78c2-4a69-9489-36e0a51b6404" (UID: "1ba9b772-78c2-4a69-9489-36e0a51b6404"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:42:05.006179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.006016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ba9b772-78c2-4a69-9489-36e0a51b6404-kube-api-access-dq8c6" (OuterVolumeSpecName: "kube-api-access-dq8c6") pod "1ba9b772-78c2-4a69-9489-36e0a51b6404" (UID: "1ba9b772-78c2-4a69-9489-36e0a51b6404"). InnerVolumeSpecName "kube-api-access-dq8c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:42:05.006179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.006064    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1ba9b772-78c2-4a69-9489-36e0a51b6404" (UID: "1ba9b772-78c2-4a69-9489-36e0a51b6404"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:42:05.104901 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.104854    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-webhook-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:42:05.104901 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.104891    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq8c6\" (UniqueName: \"kubernetes.io/projected/1ba9b772-78c2-4a69-9489-36e0a51b6404-kube-api-access-dq8c6\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:42:05.104901 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.104901    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1ba9b772-78c2-4a69-9489-36e0a51b6404-metrics-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:42:05.615669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.615630    2563 generic.go:356] "Generic (PLEG): container finished" podID="1ba9b772-78c2-4a69-9489-36e0a51b6404" containerID="ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9" exitCode=0
Sep 08 15:42:05.615889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.615686    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv"
Sep 08 15:42:05.615889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.615716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv" event={"ID":"1ba9b772-78c2-4a69-9489-36e0a51b6404","Type":"ContainerDied","Data":"ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9"}
Sep 08 15:42:05.615889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.615804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c598fcdf9-2x9sv" event={"ID":"1ba9b772-78c2-4a69-9489-36e0a51b6404","Type":"ContainerDied","Data":"b36f0d2a08dd20db0fe602edaf1de9e6c08ea4ddaca18cab27e0914f645899c3"}
Sep 08 15:42:05.615889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.615835    2563 scope.go:117] "RemoveContainer" containerID="ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9"
Sep 08 15:42:05.624963 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.624942    2563 scope.go:117] "RemoveContainer" containerID="ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9"
Sep 08 15:42:05.625259 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:42:05.625238    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9\": container with ID starting with ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9 not found: ID does not exist" containerID="ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9"
Sep 08 15:42:05.625370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.625265    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9"} err="failed to get container status \"ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9\": rpc error: code = NotFound desc = could not find container \"ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9\": container with ID starting with ed91745f113f84555121ffefaffd7fdc86a9c75bc270e231af4e9757ac3501d9 not found: ID does not exist"
Sep 08 15:42:05.636068 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.636030    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c598fcdf9-2x9sv"]
Sep 08 15:42:05.638101 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:05.638076    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c598fcdf9-2x9sv"]
Sep 08 15:42:06.100410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:06.100372    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ba9b772-78c2-4a69-9489-36e0a51b6404" path="/var/lib/kubelet/pods/1ba9b772-78c2-4a69-9489-36e0a51b6404/volumes"
Sep 08 15:42:27.386301 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.386253    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-v54tt"]
Sep 08 15:42:27.386683 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.386668    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ba9b772-78c2-4a69-9489-36e0a51b6404" containerName="manager"
Sep 08 15:42:27.386738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.386684    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ba9b772-78c2-4a69-9489-36e0a51b6404" containerName="manager"
Sep 08 15:42:27.386738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.386736    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ba9b772-78c2-4a69-9489-36e0a51b6404" containerName="manager"
Sep 08 15:42:27.392938 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.392915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.395346 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.395324    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5zp7x\""
Sep 08 15:42:27.395684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.395669    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 15:42:27.399509 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.399485    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-v54tt"]
Sep 08 15:42:27.502567 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.502531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-v54tt\" (UID: \"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.502729 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.502624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfbts\" (UniqueName: \"kubernetes.io/projected/3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878-kube-api-access-dfbts\") pod \"payload-processing-5b54b7fbb7-v54tt\" (UID: \"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.603164 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.603124    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfbts\" (UniqueName: \"kubernetes.io/projected/3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878-kube-api-access-dfbts\") pod \"payload-processing-5b54b7fbb7-v54tt\" (UID: \"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.603399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.603241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-v54tt\" (UID: \"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.603966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.603941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-v54tt\" (UID: \"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.624733 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.624699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfbts\" (UniqueName: \"kubernetes.io/projected/3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878-kube-api-access-dfbts\") pod \"payload-processing-5b54b7fbb7-v54tt\" (UID: \"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.704651 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.704613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:27.989306 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:27.989270    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-v54tt"]
Sep 08 15:42:27.991900 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:42:27.991871    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3b07f5_bbb4_49f5_9e06_2f91d8ba8878.slice/crio-e637f03d90186210ae767a355d7dc1a9cedb086f09a47118c261a6b71f7f3c57 WatchSource:0}: Error finding container e637f03d90186210ae767a355d7dc1a9cedb086f09a47118c261a6b71f7f3c57: Status 404 returned error can't find the container with id e637f03d90186210ae767a355d7dc1a9cedb086f09a47118c261a6b71f7f3c57
Sep 08 15:42:28.733253 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:28.733206    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt" event={"ID":"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878","Type":"ContainerStarted","Data":"e637f03d90186210ae767a355d7dc1a9cedb086f09a47118c261a6b71f7f3c57"}
Sep 08 15:42:31.758113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:31.757130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt" event={"ID":"3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878","Type":"ContainerStarted","Data":"d57afaa2268fabbee2ed0debf2de938f0985eb83e7e10aab0b5ff18df35cb754"}
Sep 08 15:42:31.758113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:31.758041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:31.774371 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:31.774279    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt" podStartSLOduration=1.863775268 podStartE2EDuration="4.774242881s" totalImagesPullingTime="2.910467613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:42:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:42:31.773606048 +0000 UTC m=+598.216798277" watchObservedRunningTime="2026-09-08 15:42:31.774242881 +0000 UTC m=+598.217435091"
Sep 08 15:42:36.787333 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:36.787227    2563 generic.go:356] "Generic (PLEG): container finished" podID="18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca" containerID="60ba7b504c30972b3b6beb7027771ac92d0a8e4aa297e98c7db6d69f196f7a7a" exitCode=0
Sep 08 15:42:36.787333 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:36.787276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" event={"ID":"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca","Type":"ContainerDied","Data":"60ba7b504c30972b3b6beb7027771ac92d0a8e4aa297e98c7db6d69f196f7a7a"}
Sep 08 15:42:36.787753 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:36.787687    2563 scope.go:117] "RemoveContainer" containerID="60ba7b504c30972b3b6beb7027771ac92d0a8e4aa297e98c7db6d69f196f7a7a"
Sep 08 15:42:36.788824 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:36.788794    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:42:36.788896 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:36.788878    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:42:36.788943 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:36.788930    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:42:37.793895 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:37.793855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" event={"ID":"18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca","Type":"ContainerStarted","Data":"dc58d976e6b831ceb3a8c969719335d330691a3061a9b0f4aa3dfdef4c077f5c"}
Sep 08 15:42:37.918557 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:37.918514    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:42:37.919798 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:37.919768    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xprr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 08 15:42:37.919907 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:37.919833    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" podUID="18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 08 15:42:38.918576 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:38.918534    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xprr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 08 15:42:38.919058 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:38.918609    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" podUID="18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 08 15:42:39.918382 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:39.918340    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xprr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 08 15:42:39.918567 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:39.918417    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" podUID="18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 08 15:42:40.259723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:40.258858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:42:40.918975 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:40.918938    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xprr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 08 15:42:40.919168 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:40.918995    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" podUID="18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 08 15:42:41.918264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:41.918228    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-xprr6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 08 15:42:41.918662 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:41.918308    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6" podUID="18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 08 15:42:42.922611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:42.922575    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:42:42.923495 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:42.923469    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-xprr6"
Sep 08 15:42:43.772362 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:43.772328    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-v54tt"
Sep 08 15:42:50.623795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.623762    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"]
Sep 08 15:42:50.626371 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.626351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.629661 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.629638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 15:42:50.629790 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.629637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 08 15:42:50.629790 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.629641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 15:42:50.629790 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.629644    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-przhh\""
Sep 08 15:42:50.640255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.640223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"]
Sep 08 15:42:50.726154 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.726115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.726359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.726165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.726359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.726190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c56r\" (UniqueName: \"kubernetes.io/projected/c98a5bad-0721-4837-ae3e-dce0bd31bde5-kube-api-access-9c56r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.726359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.726210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.726359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.726231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.726359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.726255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c98a5bad-0721-4837-ae3e-dce0bd31bde5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827335 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c56r\" (UniqueName: \"kubernetes.io/projected/c98a5bad-0721-4837-ae3e-dce0bd31bde5-kube-api-access-9c56r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827513 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c98a5bad-0721-4837-ae3e-dce0bd31bde5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827739 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827815 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.827940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.827919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.829778 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.829739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c98a5bad-0721-4837-ae3e-dce0bd31bde5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.829953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.829924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c98a5bad-0721-4837-ae3e-dce0bd31bde5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.834780 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.834755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c56r\" (UniqueName: \"kubernetes.io/projected/c98a5bad-0721-4837-ae3e-dce0bd31bde5-kube-api-access-9c56r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb\" (UID: \"c98a5bad-0721-4837-ae3e-dce0bd31bde5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:50.936784 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:50.936739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:42:51.211351 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:51.211326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"]
Sep 08 15:42:51.213665 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:42:51.213637    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc98a5bad_0721_4837_ae3e_dce0bd31bde5.slice/crio-f1b71b7e3828625a4e1705d2abada87ec796b8d191c30ec4a7a6c6a10f23cd78 WatchSource:0}: Error finding container f1b71b7e3828625a4e1705d2abada87ec796b8d191c30ec4a7a6c6a10f23cd78: Status 404 returned error can't find the container with id f1b71b7e3828625a4e1705d2abada87ec796b8d191c30ec4a7a6c6a10f23cd78
Sep 08 15:42:51.860304 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:51.860252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb" event={"ID":"c98a5bad-0721-4837-ae3e-dce0bd31bde5","Type":"ContainerStarted","Data":"f1b71b7e3828625a4e1705d2abada87ec796b8d191c30ec4a7a6c6a10f23cd78"}
Sep 08 15:42:52.867146 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:52.867105    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb" event={"ID":"c98a5bad-0721-4837-ae3e-dce0bd31bde5","Type":"ContainerStarted","Data":"fb8548a784a00d6759062334a07988cd3640c19b7b9eb0494f29b0565a85ab32"}
Sep 08 15:42:52.886116 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:42:52.886056    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb" podStartSLOduration=1.937147692 podStartE2EDuration="2.886038307s" totalImagesPullingTime="948.890615ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:42:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:42:52.883944003 +0000 UTC m=+619.327136230" watchObservedRunningTime="2026-09-08 15:42:52.886038307 +0000 UTC m=+619.329230517"
Sep 08 15:43:00.937452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:00.937371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:43:00.937452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:00.937445    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:43:00.949638 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:00.949606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:43:01.921043 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:01.921003    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb"
Sep 08 15:43:24.225892 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.225859    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"]
Sep 08 15:43:24.233349 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.233321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.235983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.235956    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 08 15:43:24.238042 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.238013    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"]
Sep 08 15:43:24.346918 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.346888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.347091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.346947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fqvj\" (UniqueName: \"kubernetes.io/projected/7b3fd397-6fb8-4e47-8875-1340948ecc71-kube-api-access-9fqvj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.347091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.347013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3fd397-6fb8-4e47-8875-1340948ecc71-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.347091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.347063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.347091 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.347080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.347240 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.347116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.447903 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.447860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3fd397-6fb8-4e47-8875-1340948ecc71-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448103 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.447921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448103 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.447945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448103 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.447994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448103 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.448044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448341 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.448105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fqvj\" (UniqueName: \"kubernetes.io/projected/7b3fd397-6fb8-4e47-8875-1340948ecc71-kube-api-access-9fqvj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448502 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.448479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448586 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.448470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.448586 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.448525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.450232 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.450209    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b3fd397-6fb8-4e47-8875-1340948ecc71-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.450616 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.450593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b3fd397-6fb8-4e47-8875-1340948ecc71-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.456978 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.456953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fqvj\" (UniqueName: \"kubernetes.io/projected/7b3fd397-6fb8-4e47-8875-1340948ecc71-kube-api-access-9fqvj\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2\" (UID: \"7b3fd397-6fb8-4e47-8875-1340948ecc71\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.546726 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.546627    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:24.820212 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.820182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"]
Sep 08 15:43:24.822465 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:43:24.822433    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3fd397_6fb8_4e47_8875_1340948ecc71.slice/crio-dd9143ffc87e70e98fd0720d5938dc28966d381e5442e4782c2417433ab4b0e0 WatchSource:0}: Error finding container dd9143ffc87e70e98fd0720d5938dc28966d381e5442e4782c2417433ab4b0e0: Status 404 returned error can't find the container with id dd9143ffc87e70e98fd0720d5938dc28966d381e5442e4782c2417433ab4b0e0
Sep 08 15:43:24.823855 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:24.823831    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:43:25.022326 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:25.022276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2" event={"ID":"7b3fd397-6fb8-4e47-8875-1340948ecc71","Type":"ContainerStarted","Data":"dd9143ffc87e70e98fd0720d5938dc28966d381e5442e4782c2417433ab4b0e0"}
Sep 08 15:43:26.030776 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:26.030733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2" event={"ID":"7b3fd397-6fb8-4e47-8875-1340948ecc71","Type":"ContainerStarted","Data":"46b8f9449d68eca010133a70efdb36afe7701b76eac9ce86a8983ba149cc3441"}
Sep 08 15:43:26.068359 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:26.068266    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2" podStartSLOduration=1.8684634949999999 podStartE2EDuration="2.068244177s" totalImagesPullingTime="199.780682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:26.06789622 +0000 UTC m=+652.511088453" watchObservedRunningTime="2026-09-08 15:43:26.068244177 +0000 UTC m=+652.511436388"
Sep 08 15:43:34.547202 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:34.547156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:34.547749 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:34.547261    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:34.559868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:34.559836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:35.083676 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:35.083647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2"
Sep 08 15:43:53.347819 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.347783    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"]
Sep 08 15:43:53.351615 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.351591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.360897 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.360876    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 08 15:43:53.382138 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.382092    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"]
Sep 08 15:43:53.409964 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.409920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.410139 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.409983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.410139 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.410052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.410139 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.410082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/002e5645-c39e-4884-99b3-59f2711005e0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.410139 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.410100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8vx4\" (UniqueName: \"kubernetes.io/projected/002e5645-c39e-4884-99b3-59f2711005e0-kube-api-access-h8vx4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.410139 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.410129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.511594 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.511785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.511785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.511785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/002e5645-c39e-4884-99b3-59f2711005e0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.511785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vx4\" (UniqueName: \"kubernetes.io/projected/002e5645-c39e-4884-99b3-59f2711005e0-kube-api-access-h8vx4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.511785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.512406 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.511955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.512406 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.512049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.512406 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.512127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.514008 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.513982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/002e5645-c39e-4884-99b3-59f2711005e0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.514238 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.514224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/002e5645-c39e-4884-99b3-59f2711005e0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.526823 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.526793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8vx4\" (UniqueName: \"kubernetes.io/projected/002e5645-c39e-4884-99b3-59f2711005e0-kube-api-access-h8vx4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg\" (UID: \"002e5645-c39e-4884-99b3-59f2711005e0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.662264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.662173    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:43:53.935474 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:53.935446    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"]
Sep 08 15:43:53.937823 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:43:53.937794    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002e5645_c39e_4884_99b3_59f2711005e0.slice/crio-ec88e2f19748aad778ce81d1a9365520868291d2e4ab14b7d293df2dfa0112f2 WatchSource:0}: Error finding container ec88e2f19748aad778ce81d1a9365520868291d2e4ab14b7d293df2dfa0112f2: Status 404 returned error can't find the container with id ec88e2f19748aad778ce81d1a9365520868291d2e4ab14b7d293df2dfa0112f2
Sep 08 15:43:54.164908 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:54.164880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg" event={"ID":"002e5645-c39e-4884-99b3-59f2711005e0","Type":"ContainerStarted","Data":"ec88e2f19748aad778ce81d1a9365520868291d2e4ab14b7d293df2dfa0112f2"}
Sep 08 15:43:55.171025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:55.170990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg" event={"ID":"002e5645-c39e-4884-99b3-59f2711005e0","Type":"ContainerStarted","Data":"c6ec5ea9342aa0e3a243d7a4bac5ba943b364ec36e5fa6a908de8c7f93f81eaf"}
Sep 08 15:43:55.199833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:43:55.199786    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg" podStartSLOduration=2.015833485 podStartE2EDuration="2.199767688s" totalImagesPullingTime="183.934203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:43:55.199029978 +0000 UTC m=+681.642222206" watchObservedRunningTime="2026-09-08 15:43:55.199767688 +0000 UTC m=+681.642959896"
Sep 08 15:44:03.663066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:44:03.662992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:44:03.663550 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:44:03.663107    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:44:03.682043 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:44:03.682014    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:44:04.225489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:44:04.225454    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg"
Sep 08 15:45:00.139592 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.139506    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"]
Sep 08 15:45:00.143019 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.143001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"
Sep 08 15:45:00.145353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.145329    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:45:00.150311 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.150259    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"]
Sep 08 15:45:00.154544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.154520    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"
Sep 08 15:45:00.424726 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.424595    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"]
Sep 08 15:45:00.427694 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:00.427669    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb10c3e_f7a7_4df8_bf9d_b979c5f51c5b.slice/crio-1250336efe9dda0c8429655a273aca50c419a5f5f806e15535ef527ecb53e932 WatchSource:0}: Error finding container 1250336efe9dda0c8429655a273aca50c419a5f5f806e15535ef527ecb53e932: Status 404 returned error can't find the container with id 1250336efe9dda0c8429655a273aca50c419a5f5f806e15535ef527ecb53e932
Sep 08 15:45:00.492025 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:00.491993    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5" event={"ID":"bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b","Type":"ContainerStarted","Data":"1250336efe9dda0c8429655a273aca50c419a5f5f806e15535ef527ecb53e932"}
Sep 08 15:45:04.514605 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:04.514519    2563 generic.go:356] "Generic (PLEG): container finished" podID="bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b" containerID="1cc6a7d4e38c6b9dac159ff85ddc58bde78e59a9a734a64cfa224b1123f2f936" exitCode=0
Sep 08 15:45:04.514981 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:04.514607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5" event={"ID":"bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b","Type":"ContainerDied","Data":"1cc6a7d4e38c6b9dac159ff85ddc58bde78e59a9a734a64cfa224b1123f2f936"}
Sep 08 15:45:05.650866 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:05.650841    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"
Sep 08 15:45:06.526084 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:06.526050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5" event={"ID":"bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b","Type":"ContainerDied","Data":"1250336efe9dda0c8429655a273aca50c419a5f5f806e15535ef527ecb53e932"}
Sep 08 15:45:06.526084 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:06.526087    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814705-9qrm5"
Sep 08 15:45:06.526327 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:06.526100    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1250336efe9dda0c8429655a273aca50c419a5f5f806e15535ef527ecb53e932"
Sep 08 15:45:21.922640 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.922609    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"]
Sep 08 15:45:21.923198 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.923179    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b" containerName="cleanup"
Sep 08 15:45:21.923254 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.923202    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b" containerName="cleanup"
Sep 08 15:45:21.923370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.923356    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b" containerName="cleanup"
Sep 08 15:45:21.930540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.930514    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:21.932762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.932742    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-28e012-openshift-default-dockercfg-9dgz5\""
Sep 08 15:45:21.938861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:21.938837    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"]
Sep 08 15:45:22.054699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.054663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-envoy\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.054883 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.054746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-data\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.054883 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.054836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-token\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.054883 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.054871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcs48\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-kube-api-access-gcs48\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.055370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.054949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istiod-ca-cert\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.055370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.054975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-podinfo\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.055370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.055016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-socket\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.055370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.055098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-certs\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.055370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.055150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-credential-socket\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156081 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istiod-ca-cert\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156263 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-podinfo\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156263 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-socket\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156263 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-certs\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156263 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-credential-socket\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156517 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-envoy\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156517 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-data\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156517 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-token\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156517 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcs48\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-kube-api-access-gcs48\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156608    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-certs\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-credential-socket\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156899 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istiod-ca-cert\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-data\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.156983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.156950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-socket\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.158712 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.158691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-envoy\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.158849 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.158831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-podinfo\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.163996 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.163967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-token\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.164180 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.164162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcs48\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-kube-api-access-gcs48\") pod \"e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") " pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.243745 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.243655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:22.532813 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.532786    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"]
Sep 08 15:45:22.534916 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:22.534871    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4a6f92_2453_4b66_9955_e14729c1c3fc.slice/crio-68ee13d60927623cba973081990357bf36d1493e8876081a47cfd270faa4f4cc WatchSource:0}: Error finding container 68ee13d60927623cba973081990357bf36d1493e8876081a47cfd270faa4f4cc: Status 404 returned error can't find the container with id 68ee13d60927623cba973081990357bf36d1493e8876081a47cfd270faa4f4cc
Sep 08 15:45:22.537136 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.537096    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:22.537212 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.537188    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:22.537254 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.537234    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:22.602913 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.602874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" event={"ID":"2d4a6f92-2453-4b66-9955-e14729c1c3fc","Type":"ContainerStarted","Data":"d4e784b16af8c4346712a8bb186d8dcf916e887afab96646fdb508cb99cb7ef9"}
Sep 08 15:45:22.603030 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:22.602938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" event={"ID":"2d4a6f92-2453-4b66-9955-e14729c1c3fc","Type":"ContainerStarted","Data":"68ee13d60927623cba973081990357bf36d1493e8876081a47cfd270faa4f4cc"}
Sep 08 15:45:23.632070 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:23.632011    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podStartSLOduration=2.631996492 podStartE2EDuration="2.631996492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:23.63012847 +0000 UTC m=+770.073320714" watchObservedRunningTime="2026-09-08 15:45:23.631996492 +0000 UTC m=+770.075188700"
Sep 08 15:45:24.244364 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:24.244329    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:24.250507 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:24.250476    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:24.612970 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:24.612885    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:24.613980 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:24.613961    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:27.680512 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.680477    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"]
Sep 08 15:45:27.684347 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.684329    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.686547 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.686520    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-28e012\""
Sep 08 15:45:27.686668 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.686522    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-28e012-dockercfg-6kx6z\""
Sep 08 15:45:27.695148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.694356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"]
Sep 08 15:45:27.697079 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.697052    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"]
Sep 08 15:45:27.700715 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.700694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.712221 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.712191    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"]
Sep 08 15:45:27.816272 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.816235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.816438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.816413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6xt6\" (UniqueName: \"kubernetes.io/projected/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-kube-api-access-r6xt6\") pod \"payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.816481 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.816458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2nf5\" (UniqueName: \"kubernetes.io/projected/d0cf30cd-6608-499b-9f91-a501080e5fae-kube-api-access-s2nf5\") pod \"payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.816565 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.816485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0cf30cd-6608-499b-9f91-a501080e5fae-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.917872 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.917825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.918075 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.917933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xt6\" (UniqueName: \"kubernetes.io/projected/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-kube-api-access-r6xt6\") pod \"payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.918075 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.917968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2nf5\" (UniqueName: \"kubernetes.io/projected/d0cf30cd-6608-499b-9f91-a501080e5fae-kube-api-access-s2nf5\") pod \"payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.918075 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.917994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0cf30cd-6608-499b-9f91-a501080e5fae-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.918738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.918709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.918848 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.918717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0cf30cd-6608-499b-9f91-a501080e5fae-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.926179 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.926149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6xt6\" (UniqueName: \"kubernetes.io/projected/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-kube-api-access-r6xt6\") pod \"payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:27.926270 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.926183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2nf5\" (UniqueName: \"kubernetes.io/projected/d0cf30cd-6608-499b-9f91-a501080e5fae-kube-api-access-s2nf5\") pod \"payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:27.998421 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:27.998319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:28.012502 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.012479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:28.280438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.280408    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"]
Sep 08 15:45:28.282781 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:28.282748    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0cf30cd_6608_499b_9f91_a501080e5fae.slice/crio-cd2ab0aa0c13bdabdd1cc866abfc5653bd5757c3f415ea8ddb262a3c6f38695b WatchSource:0}: Error finding container cd2ab0aa0c13bdabdd1cc866abfc5653bd5757c3f415ea8ddb262a3c6f38695b: Status 404 returned error can't find the container with id cd2ab0aa0c13bdabdd1cc866abfc5653bd5757c3f415ea8ddb262a3c6f38695b
Sep 08 15:45:28.299670 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.299639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"]
Sep 08 15:45:28.301642 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:28.301605    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b6c067_a4dd_462b_ad0c_d7e32ea4bfaa.slice/crio-aabc84615a1dca5d2d91317bfd8c97110340d01ee02db723f6893cdf4e7db2cf WatchSource:0}: Error finding container aabc84615a1dca5d2d91317bfd8c97110340d01ee02db723f6893cdf4e7db2cf: Status 404 returned error can't find the container with id aabc84615a1dca5d2d91317bfd8c97110340d01ee02db723f6893cdf4e7db2cf
Sep 08 15:45:28.634820 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.634714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8" event={"ID":"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa","Type":"ContainerStarted","Data":"25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b"}
Sep 08 15:45:28.634820 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.634785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8" event={"ID":"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa","Type":"ContainerStarted","Data":"aabc84615a1dca5d2d91317bfd8c97110340d01ee02db723f6893cdf4e7db2cf"}
Sep 08 15:45:28.634820 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.634817    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:28.636561 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.636529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz" event={"ID":"d0cf30cd-6608-499b-9f91-a501080e5fae","Type":"ContainerStarted","Data":"2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8"}
Sep 08 15:45:28.636684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.636588    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz" event={"ID":"d0cf30cd-6608-499b-9f91-a501080e5fae","Type":"ContainerStarted","Data":"cd2ab0aa0c13bdabdd1cc866abfc5653bd5757c3f415ea8ddb262a3c6f38695b"}
Sep 08 15:45:28.636684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.636622    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:28.654043 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.653990    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8" podStartSLOduration=1.6539721269999998 podStartE2EDuration="1.653972127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:28.650592076 +0000 UTC m=+775.093784333" watchObservedRunningTime="2026-09-08 15:45:28.653972127 +0000 UTC m=+775.097164327"
Sep 08 15:45:28.669400 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:28.669344    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz" podStartSLOduration=1.6693232820000001 podStartE2EDuration="1.669323282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:28.66742487 +0000 UTC m=+775.110617101" watchObservedRunningTime="2026-09-08 15:45:28.669323282 +0000 UTC m=+775.112515489"
Sep 08 15:45:30.117086 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.117053    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"]
Sep 08 15:45:30.121239 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.121220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.123399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.123377    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f1f0ac73-openshift-default-dockercfg-jl4tw\""
Sep 08 15:45:30.132459 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.132429    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"]
Sep 08 15:45:30.241204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-socket\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241437 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241324    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-data\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241437 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-podinfo\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241511 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpsg2\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-kube-api-access-wpsg2\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241553 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-certs\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241588 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-envoy\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241588 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-token\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241648 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istiod-ca-cert\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.241687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.241675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-credential-socket\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.342562 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-data\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.342719 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-podinfo\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.342719 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpsg2\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-kube-api-access-wpsg2\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.342719 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-certs\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.342719 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-envoy\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.342719 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-token\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istiod-ca-cert\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343163 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.343130    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-credential-socket\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343238 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.343193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-socket\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343238 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.343045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-certs\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343376 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.342994    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-data\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343429 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.343374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-credential-socket\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343579 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.343559    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-socket\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.343624 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.343592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istiod-ca-cert\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.345525 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.345501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-envoy\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.345738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.345707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-podinfo\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.351516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.351492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-token\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.352023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.352003    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpsg2\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-kube-api-access-wpsg2\") pod \"e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") " pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.433933 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.433902    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:30.713874 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.713845    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"]
Sep 08 15:45:30.716461 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:30.716428    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff05ce2_1d59_4b5f_94e3_9412165aad93.slice/crio-e0f51577ae9fb8955bce6cc71d1a207cc1ba847bd5e0c48e0388cf269242c33b WatchSource:0}: Error finding container e0f51577ae9fb8955bce6cc71d1a207cc1ba847bd5e0c48e0388cf269242c33b: Status 404 returned error can't find the container with id e0f51577ae9fb8955bce6cc71d1a207cc1ba847bd5e0c48e0388cf269242c33b
Sep 08 15:45:30.718760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.718725    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:30.718861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.718813    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:30.718940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:30.718859    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:31.653255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:31.653220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" event={"ID":"8ff05ce2-1d59-4b5f-94e3-9412165aad93","Type":"ContainerStarted","Data":"814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"}
Sep 08 15:45:31.653667 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:31.653272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" event={"ID":"8ff05ce2-1d59-4b5f-94e3-9412165aad93","Type":"ContainerStarted","Data":"e0f51577ae9fb8955bce6cc71d1a207cc1ba847bd5e0c48e0388cf269242c33b"}
Sep 08 15:45:31.672411 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:31.672353    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podStartSLOduration=1.672335233 podStartE2EDuration="1.672335233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:31.670613732 +0000 UTC m=+778.113805960" watchObservedRunningTime="2026-09-08 15:45:31.672335233 +0000 UTC m=+778.115527442"
Sep 08 15:45:32.434142 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.434098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:32.439365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.439338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:32.661660 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.661627    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:32.671821 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.671793    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:32.897451 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.897363    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"]
Sep 08 15:45:32.902108 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.902079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.904535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.904503    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-36c51d-openshift-default-dockercfg-c64j2\""
Sep 08 15:45:32.912240 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.912214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"]
Sep 08 15:45:32.971329 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-podinfo\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971520 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-credential-socket\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971520 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-data\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971605 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istiod-ca-cert\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971605 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-envoy\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971686 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-socket\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971733 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwhhd\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-kube-api-access-qwhhd\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971780 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-certs\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:32.971851 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:32.971828    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-token\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073079 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-credential-socket\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073079 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-data\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istiod-ca-cert\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-envoy\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-socket\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwhhd\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-kube-api-access-qwhhd\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-certs\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-token\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.073360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-podinfo\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.074001 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.073972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-data\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.074213 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.074156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-socket\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.074213 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.074164    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-certs\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.074427 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.074359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-credential-socket\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.074745 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.074722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istiod-ca-cert\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.076302 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.076258    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-envoy\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.076617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.076596    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-podinfo\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.081140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.081119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-token\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.081393 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.081369    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwhhd\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-kube-api-access-qwhhd\") pod \"e2e-autores-36c51d-openshift-default-589558d68b-njsdr\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") " pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.216146 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.216114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:33.518039 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.518007    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"]
Sep 08 15:45:33.519774 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:33.519749    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe587bb_ca16_4a35_b7f5_98f546c68d64.slice/crio-0e7203b132027a2daadaca3262610a2e5fa6d2fdb667ddef3a9a41eb2b61420b WatchSource:0}: Error finding container 0e7203b132027a2daadaca3262610a2e5fa6d2fdb667ddef3a9a41eb2b61420b: Status 404 returned error can't find the container with id 0e7203b132027a2daadaca3262610a2e5fa6d2fdb667ddef3a9a41eb2b61420b
Sep 08 15:45:33.522760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.522717    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:33.522850 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.522816    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:33.522894 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.522863    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:33.671658 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.671617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" event={"ID":"ffe587bb-ca16-4a35-b7f5-98f546c68d64","Type":"ContainerStarted","Data":"f4ac41293262fcffdfe28224481c5250c19b7d69b088af6513a0a26e05bc3030"}
Sep 08 15:45:33.672125 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.671676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" event={"ID":"ffe587bb-ca16-4a35-b7f5-98f546c68d64","Type":"ContainerStarted","Data":"0e7203b132027a2daadaca3262610a2e5fa6d2fdb667ddef3a9a41eb2b61420b"}
Sep 08 15:45:33.692654 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:33.692596    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podStartSLOduration=1.692577837 podStartE2EDuration="1.692577837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:33.689900043 +0000 UTC m=+780.133092276" watchObservedRunningTime="2026-09-08 15:45:33.692577837 +0000 UTC m=+780.135770048"
Sep 08 15:45:34.217010 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:34.216977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:34.218711 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:34.218688    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:45:34.218826 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:34.218738    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:45:35.221467 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:35.221430    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:35.681129 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:35.681096    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:35.682015 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:35.681996    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:37.461311 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.459181    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"]
Sep 08 15:45:37.469315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.468368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.471723 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.471694    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-36c51d\""
Sep 08 15:45:37.472994 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.472971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"]
Sep 08 15:45:37.620601 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.620562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f344958f-df88-4a01-a3e4-debc83a51bae-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.620795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.620611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm7nw\" (UniqueName: \"kubernetes.io/projected/f344958f-df88-4a01-a3e4-debc83a51bae-kube-api-access-rm7nw\") pod \"payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.721855 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.721763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f344958f-df88-4a01-a3e4-debc83a51bae-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.721855 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.721814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm7nw\" (UniqueName: \"kubernetes.io/projected/f344958f-df88-4a01-a3e4-debc83a51bae-kube-api-access-rm7nw\") pod \"payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.722520 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.722501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f344958f-df88-4a01-a3e4-debc83a51bae-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.732642 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.732616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm7nw\" (UniqueName: \"kubernetes.io/projected/f344958f-df88-4a01-a3e4-debc83a51bae-kube-api-access-rm7nw\") pod \"payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:37.781738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:37.781685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:38.062503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:38.062471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"]
Sep 08 15:45:38.065462 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:45:38.065433    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf344958f_df88_4a01_a3e4_debc83a51bae.slice/crio-8ffcac83194eff9a9f74b65d65bf6f55e6eafb4376052ba63fd62e9cda6e61f5 WatchSource:0}: Error finding container 8ffcac83194eff9a9f74b65d65bf6f55e6eafb4376052ba63fd62e9cda6e61f5: Status 404 returned error can't find the container with id 8ffcac83194eff9a9f74b65d65bf6f55e6eafb4376052ba63fd62e9cda6e61f5
Sep 08 15:45:38.698945 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:38.698907    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5" event={"ID":"f344958f-df88-4a01-a3e4-debc83a51bae","Type":"ContainerStarted","Data":"c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904"}
Sep 08 15:45:38.699414 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:38.698968    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5" event={"ID":"f344958f-df88-4a01-a3e4-debc83a51bae","Type":"ContainerStarted","Data":"8ffcac83194eff9a9f74b65d65bf6f55e6eafb4376052ba63fd62e9cda6e61f5"}
Sep 08 15:45:38.699414 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:38.699114    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:38.717668 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:38.717611    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5" podStartSLOduration=1.717594265 podStartE2EDuration="1.717594265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:38.714906393 +0000 UTC m=+785.158098614" watchObservedRunningTime="2026-09-08 15:45:38.717594265 +0000 UTC m=+785.160786471"
Sep 08 15:45:39.644478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:39.644447    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:45:39.644678 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:39.644515    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:45:40.654450 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:40.654411    2563 patch_prober.go:28] interesting pod/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 15:45:40.654843 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:40.654487    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 15:45:41.727668 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:41.727634    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerID="d4e784b16af8c4346712a8bb186d8dcf916e887afab96646fdb508cb99cb7ef9" exitCode=0
Sep 08 15:45:41.728164 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:41.727706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" event={"ID":"2d4a6f92-2453-4b66-9955-e14729c1c3fc","Type":"ContainerDied","Data":"d4e784b16af8c4346712a8bb186d8dcf916e887afab96646fdb508cb99cb7ef9"}
Sep 08 15:45:41.728164 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:41.728139    2563 scope.go:117] "RemoveContainer" containerID="d4e784b16af8c4346712a8bb186d8dcf916e887afab96646fdb508cb99cb7ef9"
Sep 08 15:45:41.729311 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:41.729273    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:41.729376 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:41.729353    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:41.729414 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:41.729387    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:42.735900 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:42.735866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" event={"ID":"2d4a6f92-2453-4b66-9955-e14729c1c3fc","Type":"ContainerStarted","Data":"c1dfcf90569826a1ff3aaf4a6857edfeafae0d42397a74d31aad25177956cda6"}
Sep 08 15:45:43.244682 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:43.244645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:43.244682 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:43.244676    2563 patch_prober.go:28] interesting pod/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 15:45:43.244965 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:43.244719    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 15:45:44.244366 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:44.244329    2563 patch_prober.go:28] interesting pod/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 15:45:44.244955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:44.244922    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 15:45:45.244423 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:45.244372    2563 patch_prober.go:28] interesting pod/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 15:45:45.245436 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:45.244991    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 15:45:46.244409 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:46.244368    2563 patch_prober.go:28] interesting pod/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 15:45:46.244730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:46.244700    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 15:45:47.244633 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:47.244587    2563 patch_prober.go:28] interesting pod/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 08 15:45:47.244841 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:47.244661    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 08 15:45:48.250041 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:48.250005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:48.252497 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:48.252423    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:48.253953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:48.253922    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:45:49.716245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:49.716214    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:45:55.839542 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.839442    2563 generic.go:356] "Generic (PLEG): container finished" podID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerID="f4ac41293262fcffdfe28224481c5250c19b7d69b088af6513a0a26e05bc3030" exitCode=0
Sep 08 15:45:55.839542 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.839511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" event={"ID":"ffe587bb-ca16-4a35-b7f5-98f546c68d64","Type":"ContainerDied","Data":"f4ac41293262fcffdfe28224481c5250c19b7d69b088af6513a0a26e05bc3030"}
Sep 08 15:45:55.840111 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.839923    2563 scope.go:117] "RemoveContainer" containerID="f4ac41293262fcffdfe28224481c5250c19b7d69b088af6513a0a26e05bc3030"
Sep 08 15:45:55.841014 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.840983    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:55.841109 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.841070    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:55.841181 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.841118    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:55.841237 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.841203    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerID="814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d" exitCode=0
Sep 08 15:45:55.841313 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.841259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" event={"ID":"8ff05ce2-1d59-4b5f-94e3-9412165aad93","Type":"ContainerDied","Data":"814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"}
Sep 08 15:45:55.841599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.841585    2563 scope.go:117] "RemoveContainer" containerID="814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"
Sep 08 15:45:55.842588 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.842559    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:55.842679 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.842616    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:55.842679 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:55.842648    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:45:56.854504 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:56.854456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" event={"ID":"8ff05ce2-1d59-4b5f-94e3-9412165aad93","Type":"ContainerStarted","Data":"b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1"}
Sep 08 15:45:56.858416 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:56.858339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" event={"ID":"ffe587bb-ca16-4a35-b7f5-98f546c68d64","Type":"ContainerStarted","Data":"a8a7cfbdf7cb01872110fd06821ab4c2206c63249ccf21fd7426a35806d0a7a4"}
Sep 08 15:45:57.217042 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:57.217004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:45:57.217358 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:57.217336    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:45:57.217423 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:57.217386    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:45:57.434888 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:57.434847    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:45:57.435037 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:57.434920    2563 patch_prober.go:28] interesting pod/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:45:57.435037 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:57.435002    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 15:45:58.216780 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.216733    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:45:58.217206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.216790    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:45:58.435165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.435125    2563 patch_prober.go:28] interesting pod/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:45:58.435368 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.435180    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 15:45:58.708863 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.708824    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"]
Sep 08 15:45:58.711422 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.711398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:45:58.713843 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.713820    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:45:58.721834 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.721803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"]
Sep 08 15:45:58.784315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.775834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e602ae-df57-4520-b2f4-4e2bd58ad054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h\" (UID: \"c5e602ae-df57-4520-b2f4-4e2bd58ad054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:45:58.883991 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.883945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e602ae-df57-4520-b2f4-4e2bd58ad054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h\" (UID: \"c5e602ae-df57-4520-b2f4-4e2bd58ad054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:45:58.885151 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:58.885123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e602ae-df57-4520-b2f4-4e2bd58ad054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h\" (UID: \"c5e602ae-df57-4520-b2f4-4e2bd58ad054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:45:59.022379 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.022277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:45:59.216968 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.216931    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:45:59.217335 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.216987    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:45:59.341939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.341910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"]
Sep 08 15:45:59.434520 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.434480    2563 patch_prober.go:28] interesting pod/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:45:59.434809 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.434776    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 15:45:59.900589 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.900555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h" event={"ID":"c5e602ae-df57-4520-b2f4-4e2bd58ad054","Type":"ContainerStarted","Data":"709afcee51cb0b3a68639861469d3b1f0f9d426d5b1d5683a0e075b97cb897bd"}
Sep 08 15:45:59.900807 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.900794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h" event={"ID":"c5e602ae-df57-4520-b2f4-4e2bd58ad054","Type":"ContainerStarted","Data":"dd26bacb90d7d044eb8d919ebfd12d3aedbb66b5cf024dc3d3ec428d687d4668"}
Sep 08 15:45:59.930192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:45:59.930128    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h" podStartSLOduration=1.93011104 podStartE2EDuration="1.93011104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:45:59.928057627 +0000 UTC m=+806.371249854" watchObservedRunningTime="2026-09-08 15:45:59.93011104 +0000 UTC m=+806.373303250"
Sep 08 15:46:00.217218 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:00.217186    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:46:00.217692 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:00.217236    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:46:00.434410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:00.434368    2563 patch_prober.go:28] interesting pod/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:46:00.434587 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:00.434429    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 15:46:00.908799 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:00.908737    2563 generic.go:356] "Generic (PLEG): container finished" podID="c5e602ae-df57-4520-b2f4-4e2bd58ad054" containerID="709afcee51cb0b3a68639861469d3b1f0f9d426d5b1d5683a0e075b97cb897bd" exitCode=0
Sep 08 15:46:00.908977 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:00.908861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h" event={"ID":"c5e602ae-df57-4520-b2f4-4e2bd58ad054","Type":"ContainerDied","Data":"709afcee51cb0b3a68639861469d3b1f0f9d426d5b1d5683a0e075b97cb897bd"}
Sep 08 15:46:01.216856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.216818    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:46:01.217031 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.216884    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:46:01.236340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.236302    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"]
Sep 08 15:46:01.239572 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.239551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.252437 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.246657    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a057c5-dockercfg-c7d44\""
Sep 08 15:46:01.252437 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.246919    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a057c5\""
Sep 08 15:46:01.270573 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.270535    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"]
Sep 08 15:46:01.314165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.313986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450ef3d2-9f02-404b-9377-d14ca9f5bf98-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.314165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.314041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7kr6\" (UniqueName: \"kubernetes.io/projected/450ef3d2-9f02-404b-9377-d14ca9f5bf98-kube-api-access-j7kr6\") pod \"payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.414735 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.414697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7kr6\" (UniqueName: \"kubernetes.io/projected/450ef3d2-9f02-404b-9377-d14ca9f5bf98-kube-api-access-j7kr6\") pod \"payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.414961 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.414879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450ef3d2-9f02-404b-9377-d14ca9f5bf98-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.415619 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.415593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450ef3d2-9f02-404b-9377-d14ca9f5bf98-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.423479 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.423458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7kr6\" (UniqueName: \"kubernetes.io/projected/450ef3d2-9f02-404b-9377-d14ca9f5bf98-kube-api-access-j7kr6\") pod \"payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.434555 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.434529    2563 patch_prober.go:28] interesting pod/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:46:01.434648 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.434587    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 15:46:01.571915 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.571800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:01.862364 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.860501    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"]
Sep 08 15:46:01.916847 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:01.916816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn" event={"ID":"450ef3d2-9f02-404b-9377-d14ca9f5bf98","Type":"ContainerStarted","Data":"2bc15c5225fbf3c001eaf8eecb598bc400d523cba152db158c4fec13ee6be310"}
Sep 08 15:46:02.184312 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.182027    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:46:02.217394 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.217360    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 08 15:46:02.217560 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.217419    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 08 15:46:02.231148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.231114    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e602ae-df57-4520-b2f4-4e2bd58ad054-maas-api-service-ca\") pod \"c5e602ae-df57-4520-b2f4-4e2bd58ad054\" (UID: \"c5e602ae-df57-4520-b2f4-4e2bd58ad054\") "
Sep 08 15:46:02.232551 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.232512    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5e602ae-df57-4520-b2f4-4e2bd58ad054-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5e602ae-df57-4520-b2f4-4e2bd58ad054" (UID: "c5e602ae-df57-4520-b2f4-4e2bd58ad054"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:02.333066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.333018    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e602ae-df57-4520-b2f4-4e2bd58ad054-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:02.434452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.434406    2563 patch_prober.go:28] interesting pod/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 08 15:46:02.434639 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.434483    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 08 15:46:02.661908 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.661854    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:46:02.925725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.925688    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"
Sep 08 15:46:02.925934 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.925725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h" event={"ID":"c5e602ae-df57-4520-b2f4-4e2bd58ad054","Type":"ContainerDied","Data":"dd26bacb90d7d044eb8d919ebfd12d3aedbb66b5cf024dc3d3ec428d687d4668"}
Sep 08 15:46:02.925934 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.925780    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd26bacb90d7d044eb8d919ebfd12d3aedbb66b5cf024dc3d3ec428d687d4668"
Sep 08 15:46:02.927761 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.927733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn" event={"ID":"450ef3d2-9f02-404b-9377-d14ca9f5bf98","Type":"ContainerStarted","Data":"5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b"}
Sep 08 15:46:02.927905 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.927869    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:02.947143 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:02.947088    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn" podStartSLOduration=1.947072146 podStartE2EDuration="1.947072146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:02.944186626 +0000 UTC m=+809.387378856" watchObservedRunningTime="2026-09-08 15:46:02.947072146 +0000 UTC m=+809.390264382"
Sep 08 15:46:03.221482 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:03.221402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:46:03.221836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:03.221812    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:46:03.222554 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:03.222536    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:46:03.438234 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:03.438196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:46:03.439215 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:03.439198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:46:05.478997 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:05.478957    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"]
Sep 08 15:46:05.488093 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:05.488063    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f1f0ac73-c5v2h"]
Sep 08 15:46:05.612784 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:05.612751    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"]
Sep 08 15:46:05.612969 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:05.612944    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy" containerID="cri-o://b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1" gracePeriod=30
Sep 08 15:46:06.108921 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:06.108877    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e602ae-df57-4520-b2f4-4e2bd58ad054" path="/var/lib/kubelet/pods/c5e602ae-df57-4520-b2f4-4e2bd58ad054/volumes"
Sep 08 15:46:10.867924 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.867893    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:46:10.927376 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927343    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-data\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927573 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927418    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-socket\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927573 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927444    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-token\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927573 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-credential-socket\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927573 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-envoy\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927782 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927661    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpsg2\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-kube-api-access-wpsg2\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927782 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927711    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:46:10.927782 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927733    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:46:10.927782 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927740    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istiod-ca-cert\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927977 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927820    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-certs\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.927977 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:46:10.927977 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.927903    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-podinfo\") pod \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\" (UID: \"8ff05ce2-1d59-4b5f-94e3-9412165aad93\") "
Sep 08 15:46:10.928140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:46:10.928140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:10.928434 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928410    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:10.928434 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928431    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:10.928581 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928446    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:10.928581 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928456    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:10.928581 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.928468    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:10.930015 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.929993    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:46:10.930015 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.930005    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:46:10.930156 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.930108    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-kube-api-access-wpsg2" (OuterVolumeSpecName: "kube-api-access-wpsg2") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "kube-api-access-wpsg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:46:10.930156 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.930132    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ff05ce2-1d59-4b5f-94e3-9412165aad93" (UID: "8ff05ce2-1d59-4b5f-94e3-9412165aad93"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:46:10.972729 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.972698    2563 generic.go:356] "Generic (PLEG): container finished" podID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerID="b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1" exitCode=0
Sep 08 15:46:10.972902 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.972750    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"
Sep 08 15:46:10.972902 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.972768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" event={"ID":"8ff05ce2-1d59-4b5f-94e3-9412165aad93","Type":"ContainerDied","Data":"b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1"}
Sep 08 15:46:10.972902 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.972810    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j" event={"ID":"8ff05ce2-1d59-4b5f-94e3-9412165aad93","Type":"ContainerDied","Data":"e0f51577ae9fb8955bce6cc71d1a207cc1ba847bd5e0c48e0388cf269242c33b"}
Sep 08 15:46:10.972902 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.972833    2563 scope.go:117] "RemoveContainer" containerID="b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1"
Sep 08 15:46:10.982772 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.982753    2563 scope.go:117] "RemoveContainer" containerID="814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"
Sep 08 15:46:10.991007 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.990986    2563 scope.go:117] "RemoveContainer" containerID="b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1"
Sep 08 15:46:10.991262 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:46:10.991245    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1\": container with ID starting with b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1 not found: ID does not exist" containerID="b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1"
Sep 08 15:46:10.991315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.991272    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1"} err="failed to get container status \"b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1\": rpc error: code = NotFound desc = could not find container \"b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1\": container with ID starting with b6ab48779ae0f6c3675bfbb8f3730472b2cfa5755718ab53da19f766d51ad6e1 not found: ID does not exist"
Sep 08 15:46:10.991315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.991310    2563 scope.go:117] "RemoveContainer" containerID="814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"
Sep 08 15:46:10.991638 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:46:10.991612    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d\": container with ID starting with 814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d not found: ID does not exist" containerID="814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"
Sep 08 15:46:10.991738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.991646    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d"} err="failed to get container status \"814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d\": rpc error: code = NotFound desc = could not find container \"814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d\": container with ID starting with 814cbdc27b90823249ab560ff6f54d62d1f1124f0f8b242969480ec82ff6c00d not found: ID does not exist"
Sep 08 15:46:10.994490 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:10.994464    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"]
Sep 08 15:46:11.002704 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:11.002680    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f1f0ac73-openshift-default-5944485d9d-n2t4j"]
Sep 08 15:46:11.029715 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:11.029691    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:11.029715 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:11.029713    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:11.029853 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:11.029728    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpsg2\" (UniqueName: \"kubernetes.io/projected/8ff05ce2-1d59-4b5f-94e3-9412165aad93-kube-api-access-wpsg2\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:11.029853 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:11.029740    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ff05ce2-1d59-4b5f-94e3-9412165aad93-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:12.101305 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:12.101266    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" path="/var/lib/kubelet/pods/8ff05ce2-1d59-4b5f-94e3-9412165aad93/volumes"
Sep 08 15:46:13.492822 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.492788    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"]
Sep 08 15:46:13.493362 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493341    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy"
Sep 08 15:46:13.493362 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493359    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy"
Sep 08 15:46:13.493362 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493368    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e602ae-df57-4520-b2f4-4e2bd58ad054" containerName="revoke-keys"
Sep 08 15:46:13.493609 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493376    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e602ae-df57-4520-b2f4-4e2bd58ad054" containerName="revoke-keys"
Sep 08 15:46:13.493609 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493433    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy"
Sep 08 15:46:13.493609 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493450    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy"
Sep 08 15:46:13.493777 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493613    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy"
Sep 08 15:46:13.493777 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493630    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e602ae-df57-4520-b2f4-4e2bd58ad054" containerName="revoke-keys"
Sep 08 15:46:13.493777 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.493663    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff05ce2-1d59-4b5f-94e3-9412165aad93" containerName="istio-proxy"
Sep 08 15:46:13.497982 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.497967    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.500532 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.500509    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-13162f1c\""
Sep 08 15:46:13.500532 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.500525    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-13162f1c-dockercfg-mw8rp\""
Sep 08 15:46:13.504328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.504307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"]
Sep 08 15:46:13.565294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.565250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49zfk\" (UniqueName: \"kubernetes.io/projected/2c2786be-95ed-49aa-8557-03d89e8e7bad-kube-api-access-49zfk\") pod \"payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.565488 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.565352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c2786be-95ed-49aa-8557-03d89e8e7bad-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.666594 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.666555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49zfk\" (UniqueName: \"kubernetes.io/projected/2c2786be-95ed-49aa-8557-03d89e8e7bad-kube-api-access-49zfk\") pod \"payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.666594 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.666601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c2786be-95ed-49aa-8557-03d89e8e7bad-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.667332 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.667279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c2786be-95ed-49aa-8557-03d89e8e7bad-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.674245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.674217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49zfk\" (UniqueName: \"kubernetes.io/projected/2c2786be-95ed-49aa-8557-03d89e8e7bad-kube-api-access-49zfk\") pod \"payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.810258 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.810153    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:13.935439 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:13.935408    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:46:14.092818 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:46:14.092776    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2786be_95ed_49aa_8557_03d89e8e7bad.slice/crio-e412829f6eba5e552de66a958464e463fb12147f2fd0624cca2b5706e61b7c2b WatchSource:0}: Error finding container e412829f6eba5e552de66a958464e463fb12147f2fd0624cca2b5706e61b7c2b: Status 404 returned error can't find the container with id e412829f6eba5e552de66a958464e463fb12147f2fd0624cca2b5706e61b7c2b
Sep 08 15:46:14.102140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:14.102116    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"]
Sep 08 15:46:14.997999 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:14.997957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d" event={"ID":"2c2786be-95ed-49aa-8557-03d89e8e7bad","Type":"ContainerStarted","Data":"fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33"}
Sep 08 15:46:14.998397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:14.998034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d" event={"ID":"2c2786be-95ed-49aa-8557-03d89e8e7bad","Type":"ContainerStarted","Data":"e412829f6eba5e552de66a958464e463fb12147f2fd0624cca2b5706e61b7c2b"}
Sep 08 15:46:14.998397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:14.998067    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:15.014585 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:15.014534    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d" podStartSLOduration=2.014514293 podStartE2EDuration="2.014514293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:46:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:15.012798657 +0000 UTC m=+821.455990882" watchObservedRunningTime="2026-09-08 15:46:15.014514293 +0000 UTC m=+821.457706502"
Sep 08 15:46:26.005767 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:26.005733    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:37.661849 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.661810    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"]
Sep 08 15:46:37.665199 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.665180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:37.667556 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.667530    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:46:37.672694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.672669    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"]
Sep 08 15:46:37.714274 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.714235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca011d-97ae-448c-8132-d4b6031ea45b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv\" (UID: \"6bca011d-97ae-448c-8132-d4b6031ea45b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:37.815721 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.815678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca011d-97ae-448c-8132-d4b6031ea45b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv\" (UID: \"6bca011d-97ae-448c-8132-d4b6031ea45b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:37.816364 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.816344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca011d-97ae-448c-8132-d4b6031ea45b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv\" (UID: \"6bca011d-97ae-448c-8132-d4b6031ea45b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:37.977259 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:37.977225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:38.246306 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:38.246267    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"]
Sep 08 15:46:38.248094 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:46:38.248065    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bca011d_97ae_448c_8132_d4b6031ea45b.slice/crio-8c26853db920ad812a28a1a116d68252b8b1db0943e29f1a3eb6c411f983319e WatchSource:0}: Error finding container 8c26853db920ad812a28a1a116d68252b8b1db0943e29f1a3eb6c411f983319e: Status 404 returned error can't find the container with id 8c26853db920ad812a28a1a116d68252b8b1db0943e29f1a3eb6c411f983319e
Sep 08 15:46:39.128264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:39.128232    2563 generic.go:356] "Generic (PLEG): container finished" podID="6bca011d-97ae-448c-8132-d4b6031ea45b" containerID="dd743781fc24b8d503e231a11a138c9fee63c8e27be2e181137610adc2d35125" exitCode=0
Sep 08 15:46:39.128737 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:39.128323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv" event={"ID":"6bca011d-97ae-448c-8132-d4b6031ea45b","Type":"ContainerDied","Data":"dd743781fc24b8d503e231a11a138c9fee63c8e27be2e181137610adc2d35125"}
Sep 08 15:46:39.128737 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:39.128382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv" event={"ID":"6bca011d-97ae-448c-8132-d4b6031ea45b","Type":"ContainerStarted","Data":"8c26853db920ad812a28a1a116d68252b8b1db0943e29f1a3eb6c411f983319e"}
Sep 08 15:46:40.276834 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:40.276810    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:40.339603 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:40.339571    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca011d-97ae-448c-8132-d4b6031ea45b-maas-api-service-ca\") pod \"6bca011d-97ae-448c-8132-d4b6031ea45b\" (UID: \"6bca011d-97ae-448c-8132-d4b6031ea45b\") "
Sep 08 15:46:40.339985 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:40.339960    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bca011d-97ae-448c-8132-d4b6031ea45b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6bca011d-97ae-448c-8132-d4b6031ea45b" (UID: "6bca011d-97ae-448c-8132-d4b6031ea45b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:40.441240 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:40.441199    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bca011d-97ae-448c-8132-d4b6031ea45b-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:41.140494 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:41.140463    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"
Sep 08 15:46:41.140678 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:41.140499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv" event={"ID":"6bca011d-97ae-448c-8132-d4b6031ea45b","Type":"ContainerDied","Data":"8c26853db920ad812a28a1a116d68252b8b1db0943e29f1a3eb6c411f983319e"}
Sep 08 15:46:41.140678 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:41.140549    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c26853db920ad812a28a1a116d68252b8b1db0943e29f1a3eb6c411f983319e"
Sep 08 15:46:44.208710 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.208669    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"]
Sep 08 15:46:44.209206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.208928    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d" podUID="2c2786be-95ed-49aa-8557-03d89e8e7bad" containerName="payload-pre-processing" containerID="cri-o://fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33" gracePeriod=30
Sep 08 15:46:44.345258 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.345224    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"]
Sep 08 15:46:44.350409 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.350379    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-13162f1c-8wxqv"]
Sep 08 15:46:44.476062 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.476026    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:44.581106 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.581070    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49zfk\" (UniqueName: \"kubernetes.io/projected/2c2786be-95ed-49aa-8557-03d89e8e7bad-kube-api-access-49zfk\") pod \"2c2786be-95ed-49aa-8557-03d89e8e7bad\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") "
Sep 08 15:46:44.581318 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.581122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c2786be-95ed-49aa-8557-03d89e8e7bad-plugins-config-volume\") pod \"2c2786be-95ed-49aa-8557-03d89e8e7bad\" (UID: \"2c2786be-95ed-49aa-8557-03d89e8e7bad\") "
Sep 08 15:46:44.581596 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.581565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c2786be-95ed-49aa-8557-03d89e8e7bad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2c2786be-95ed-49aa-8557-03d89e8e7bad" (UID: "2c2786be-95ed-49aa-8557-03d89e8e7bad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:46:44.583294 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.583260    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c2786be-95ed-49aa-8557-03d89e8e7bad-kube-api-access-49zfk" (OuterVolumeSpecName: "kube-api-access-49zfk") pod "2c2786be-95ed-49aa-8557-03d89e8e7bad" (UID: "2c2786be-95ed-49aa-8557-03d89e8e7bad"). InnerVolumeSpecName "kube-api-access-49zfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:46:44.682303 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.682258    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49zfk\" (UniqueName: \"kubernetes.io/projected/2c2786be-95ed-49aa-8557-03d89e8e7bad-kube-api-access-49zfk\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:44.682495 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:44.682337    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c2786be-95ed-49aa-8557-03d89e8e7bad-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:46:45.166356 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.166322    2563 generic.go:356] "Generic (PLEG): container finished" podID="2c2786be-95ed-49aa-8557-03d89e8e7bad" containerID="fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33" exitCode=0
Sep 08 15:46:45.166535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.166379    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"
Sep 08 15:46:45.166535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.166405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d" event={"ID":"2c2786be-95ed-49aa-8557-03d89e8e7bad","Type":"ContainerDied","Data":"fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33"}
Sep 08 15:46:45.166535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.166466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d" event={"ID":"2c2786be-95ed-49aa-8557-03d89e8e7bad","Type":"ContainerDied","Data":"e412829f6eba5e552de66a958464e463fb12147f2fd0624cca2b5706e61b7c2b"}
Sep 08 15:46:45.166535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.166496    2563 scope.go:117] "RemoveContainer" containerID="fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33"
Sep 08 15:46:45.176979 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.176956    2563 scope.go:117] "RemoveContainer" containerID="fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33"
Sep 08 15:46:45.177372 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:46:45.177277    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33\": container with ID starting with fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33 not found: ID does not exist" containerID="fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33"
Sep 08 15:46:45.177372 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.177334    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33"} err="failed to get container status \"fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33\": rpc error: code = NotFound desc = could not find container \"fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33\": container with ID starting with fb72b65a014b909647bc655395db9e3252258918e69397ccdb171d72d8cbec33 not found: ID does not exist"
Sep 08 15:46:45.190703 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.190665    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"]
Sep 08 15:46:45.194862 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:45.194835    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-13162f1c-5b59c87c6b-24p6d"]
Sep 08 15:46:46.100689 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:46.100653    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c2786be-95ed-49aa-8557-03d89e8e7bad" path="/var/lib/kubelet/pods/2c2786be-95ed-49aa-8557-03d89e8e7bad/volumes"
Sep 08 15:46:46.101077 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:46.100986    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bca011d-97ae-448c-8132-d4b6031ea45b" path="/var/lib/kubelet/pods/6bca011d-97ae-448c-8132-d4b6031ea45b/volumes"
Sep 08 15:46:49.696644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.696606    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"]
Sep 08 15:46:49.697053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.697038    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2786be-95ed-49aa-8557-03d89e8e7bad" containerName="payload-pre-processing"
Sep 08 15:46:49.697098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.697054    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2786be-95ed-49aa-8557-03d89e8e7bad" containerName="payload-pre-processing"
Sep 08 15:46:49.697098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.697085    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bca011d-97ae-448c-8132-d4b6031ea45b" containerName="revoke-keys"
Sep 08 15:46:49.697098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.697091    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bca011d-97ae-448c-8132-d4b6031ea45b" containerName="revoke-keys"
Sep 08 15:46:49.697196 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.697149    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bca011d-97ae-448c-8132-d4b6031ea45b" containerName="revoke-keys"
Sep 08 15:46:49.697196 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.697158    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c2786be-95ed-49aa-8557-03d89e8e7bad" containerName="payload-pre-processing"
Sep 08 15:46:49.701650 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.701631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:49.704519 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.704493    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ddd2c784-dockercfg-2hff9\""
Sep 08 15:46:49.704689 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.704526    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ddd2c784\""
Sep 08 15:46:49.711889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.711859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"]
Sep 08 15:46:49.837080 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.837036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz6sp\" (UniqueName: \"kubernetes.io/projected/c4180c5f-7602-44e9-9914-c55bf747696e-kube-api-access-bz6sp\") pod \"payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:49.837268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.837196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4180c5f-7602-44e9-9914-c55bf747696e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:49.938370 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.938277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4180c5f-7602-44e9-9914-c55bf747696e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:49.938565 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.938435    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz6sp\" (UniqueName: \"kubernetes.io/projected/c4180c5f-7602-44e9-9914-c55bf747696e-kube-api-access-bz6sp\") pod \"payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:49.939118 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.939096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4180c5f-7602-44e9-9914-c55bf747696e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:49.946997 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:49.946912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz6sp\" (UniqueName: \"kubernetes.io/projected/c4180c5f-7602-44e9-9914-c55bf747696e-kube-api-access-bz6sp\") pod \"payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:50.015421 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:50.015384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:50.287420 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:50.287391    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"]
Sep 08 15:46:50.289129 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:46:50.289097    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4180c5f_7602_44e9_9914_c55bf747696e.slice/crio-2d12285f9b57aea1cdc6a37c0c7794c20978b4a293b198b209ebf1ded51570a2 WatchSource:0}: Error finding container 2d12285f9b57aea1cdc6a37c0c7794c20978b4a293b198b209ebf1ded51570a2: Status 404 returned error can't find the container with id 2d12285f9b57aea1cdc6a37c0c7794c20978b4a293b198b209ebf1ded51570a2
Sep 08 15:46:51.207953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:51.207914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c" event={"ID":"c4180c5f-7602-44e9-9914-c55bf747696e","Type":"ContainerStarted","Data":"4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02"}
Sep 08 15:46:51.208365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:51.208014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c" event={"ID":"c4180c5f-7602-44e9-9914-c55bf747696e","Type":"ContainerStarted","Data":"2d12285f9b57aea1cdc6a37c0c7794c20978b4a293b198b209ebf1ded51570a2"}
Sep 08 15:46:51.208365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:51.208047    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:46:51.224445 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:46:51.224387    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c" podStartSLOduration=2.22437031 podStartE2EDuration="2.22437031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:46:51.222616545 +0000 UTC m=+857.665808761" watchObservedRunningTime="2026-09-08 15:46:51.22437031 +0000 UTC m=+857.667562520"
Sep 08 15:47:02.215739 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:02.215705    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:47:13.643744 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.643706    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"]
Sep 08 15:47:13.647627 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.647607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:13.650472 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.650442    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:47:13.653833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.653801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"]
Sep 08 15:47:13.688201 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.688152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3035dde8-4234-41e9-a51a-dde8bc23f794-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc\" (UID: \"3035dde8-4234-41e9-a51a-dde8bc23f794\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:13.789366 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.789331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3035dde8-4234-41e9-a51a-dde8bc23f794-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc\" (UID: \"3035dde8-4234-41e9-a51a-dde8bc23f794\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:13.790011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.789988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3035dde8-4234-41e9-a51a-dde8bc23f794-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc\" (UID: \"3035dde8-4234-41e9-a51a-dde8bc23f794\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:13.961249 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:13.961212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:14.234462 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:14.234432    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"]
Sep 08 15:47:14.236435 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:47:14.236403    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3035dde8_4234_41e9_a51a_dde8bc23f794.slice/crio-c33a2fc1885b55bdbc0f3ed4fc3a9c137b9cd7f8a4365de26408e89ed70044df WatchSource:0}: Error finding container c33a2fc1885b55bdbc0f3ed4fc3a9c137b9cd7f8a4365de26408e89ed70044df: Status 404 returned error can't find the container with id c33a2fc1885b55bdbc0f3ed4fc3a9c137b9cd7f8a4365de26408e89ed70044df
Sep 08 15:47:14.336235 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:14.336200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc" event={"ID":"3035dde8-4234-41e9-a51a-dde8bc23f794","Type":"ContainerStarted","Data":"c33a2fc1885b55bdbc0f3ed4fc3a9c137b9cd7f8a4365de26408e89ed70044df"}
Sep 08 15:47:15.344197 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:15.344116    2563 generic.go:356] "Generic (PLEG): container finished" podID="3035dde8-4234-41e9-a51a-dde8bc23f794" containerID="98a070e9bb7360e300dde3139b3ca4c20184e544340686663fc8d52e55e4091a" exitCode=0
Sep 08 15:47:15.344656 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:15.344202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc" event={"ID":"3035dde8-4234-41e9-a51a-dde8bc23f794","Type":"ContainerDied","Data":"98a070e9bb7360e300dde3139b3ca4c20184e544340686663fc8d52e55e4091a"}
Sep 08 15:47:16.514051 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:16.514025    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:16.620506 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:16.620417    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3035dde8-4234-41e9-a51a-dde8bc23f794-maas-api-service-ca\") pod \"3035dde8-4234-41e9-a51a-dde8bc23f794\" (UID: \"3035dde8-4234-41e9-a51a-dde8bc23f794\") "
Sep 08 15:47:16.620774 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:16.620750    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3035dde8-4234-41e9-a51a-dde8bc23f794-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3035dde8-4234-41e9-a51a-dde8bc23f794" (UID: "3035dde8-4234-41e9-a51a-dde8bc23f794"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:16.721507 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:16.721464    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3035dde8-4234-41e9-a51a-dde8bc23f794-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:17.356885 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:17.356852    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"
Sep 08 15:47:17.357089 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:17.356897    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc" event={"ID":"3035dde8-4234-41e9-a51a-dde8bc23f794","Type":"ContainerDied","Data":"c33a2fc1885b55bdbc0f3ed4fc3a9c137b9cd7f8a4365de26408e89ed70044df"}
Sep 08 15:47:17.357089 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:17.356943    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c33a2fc1885b55bdbc0f3ed4fc3a9c137b9cd7f8a4365de26408e89ed70044df"
Sep 08 15:47:19.613617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.613577    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"]
Sep 08 15:47:19.614445 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.613893    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c" podUID="c4180c5f-7602-44e9-9914-c55bf747696e" containerName="payload-pre-processing" containerID="cri-o://4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02" gracePeriod=30
Sep 08 15:47:19.772975 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.772939    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"]
Sep 08 15:47:19.778207 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.778175    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddd2c784-vtzmc"]
Sep 08 15:47:19.876490 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.876428    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:47:19.957727 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.957693    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4180c5f-7602-44e9-9914-c55bf747696e-plugins-config-volume\") pod \"c4180c5f-7602-44e9-9914-c55bf747696e\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") "
Sep 08 15:47:19.957897 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.957804    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz6sp\" (UniqueName: \"kubernetes.io/projected/c4180c5f-7602-44e9-9914-c55bf747696e-kube-api-access-bz6sp\") pod \"c4180c5f-7602-44e9-9914-c55bf747696e\" (UID: \"c4180c5f-7602-44e9-9914-c55bf747696e\") "
Sep 08 15:47:19.958092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.958069    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4180c5f-7602-44e9-9914-c55bf747696e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4180c5f-7602-44e9-9914-c55bf747696e" (UID: "c4180c5f-7602-44e9-9914-c55bf747696e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:19.958210 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.958190    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4180c5f-7602-44e9-9914-c55bf747696e-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:19.959924 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:19.959887    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4180c5f-7602-44e9-9914-c55bf747696e-kube-api-access-bz6sp" (OuterVolumeSpecName: "kube-api-access-bz6sp") pod "c4180c5f-7602-44e9-9914-c55bf747696e" (UID: "c4180c5f-7602-44e9-9914-c55bf747696e"). InnerVolumeSpecName "kube-api-access-bz6sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:20.059100 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.059057    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz6sp\" (UniqueName: \"kubernetes.io/projected/c4180c5f-7602-44e9-9914-c55bf747696e-kube-api-access-bz6sp\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:20.102096 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.102060    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3035dde8-4234-41e9-a51a-dde8bc23f794" path="/var/lib/kubelet/pods/3035dde8-4234-41e9-a51a-dde8bc23f794/volumes"
Sep 08 15:47:20.374635 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.374599    2563 generic.go:356] "Generic (PLEG): container finished" podID="c4180c5f-7602-44e9-9914-c55bf747696e" containerID="4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02" exitCode=0
Sep 08 15:47:20.374833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.374642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c" event={"ID":"c4180c5f-7602-44e9-9914-c55bf747696e","Type":"ContainerDied","Data":"4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02"}
Sep 08 15:47:20.374833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.374663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"
Sep 08 15:47:20.374833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.374683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c" event={"ID":"c4180c5f-7602-44e9-9914-c55bf747696e","Type":"ContainerDied","Data":"2d12285f9b57aea1cdc6a37c0c7794c20978b4a293b198b209ebf1ded51570a2"}
Sep 08 15:47:20.374833 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.374704    2563 scope.go:117] "RemoveContainer" containerID="4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02"
Sep 08 15:47:20.383861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.383840    2563 scope.go:117] "RemoveContainer" containerID="4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02"
Sep 08 15:47:20.384153 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:47:20.384134    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02\": container with ID starting with 4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02 not found: ID does not exist" containerID="4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02"
Sep 08 15:47:20.384206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.384162    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02"} err="failed to get container status \"4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02\": rpc error: code = NotFound desc = could not find container \"4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02\": container with ID starting with 4e48bd9f96c30ab55214ede04fdcc60f5222cbcfcec4b7d3f6f3fd8f3012af02 not found: ID does not exist"
Sep 08 15:47:20.397605 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.397573    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"]
Sep 08 15:47:20.405853 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:20.405821    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddd2c784-78b6bf9dd6-rgq5c"]
Sep 08 15:47:22.100716 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:22.100682    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4180c5f-7602-44e9-9914-c55bf747696e" path="/var/lib/kubelet/pods/c4180c5f-7602-44e9-9914-c55bf747696e/volumes"
Sep 08 15:47:25.836351 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836238    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"]
Sep 08 15:47:25.836758 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3035dde8-4234-41e9-a51a-dde8bc23f794" containerName="revoke-keys"
Sep 08 15:47:25.836758 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836722    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3035dde8-4234-41e9-a51a-dde8bc23f794" containerName="revoke-keys"
Sep 08 15:47:25.836758 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836731    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4180c5f-7602-44e9-9914-c55bf747696e" containerName="payload-pre-processing"
Sep 08 15:47:25.836758 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836737    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4180c5f-7602-44e9-9914-c55bf747696e" containerName="payload-pre-processing"
Sep 08 15:47:25.836889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836812    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4180c5f-7602-44e9-9914-c55bf747696e" containerName="payload-pre-processing"
Sep 08 15:47:25.836889 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.836823    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3035dde8-4234-41e9-a51a-dde8bc23f794" containerName="revoke-keys"
Sep 08 15:47:25.841382 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.841361    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:25.843934 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.843910    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a82afc10-dockercfg-5vb89\""
Sep 08 15:47:25.844053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.843910    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a82afc10\""
Sep 08 15:47:25.851118 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.851091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"]
Sep 08 15:47:25.919951 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.919910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a461b46-658e-49ba-ae2c-7a68668dbdd9-plugins-config-volume\") pod \"payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") " pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:25.920297 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:25.920254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6426\" (UniqueName: \"kubernetes.io/projected/1a461b46-658e-49ba-ae2c-7a68668dbdd9-kube-api-access-k6426\") pod \"payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") " pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:26.021912 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:26.021858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a461b46-658e-49ba-ae2c-7a68668dbdd9-plugins-config-volume\") pod \"payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") " pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:26.021912 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:26.021925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6426\" (UniqueName: \"kubernetes.io/projected/1a461b46-658e-49ba-ae2c-7a68668dbdd9-kube-api-access-k6426\") pod \"payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") " pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:26.022618 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:26.022598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a461b46-658e-49ba-ae2c-7a68668dbdd9-plugins-config-volume\") pod \"payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") " pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:26.030657 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:26.030624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6426\" (UniqueName: \"kubernetes.io/projected/1a461b46-658e-49ba-ae2c-7a68668dbdd9-kube-api-access-k6426\") pod \"payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") " pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:26.154908 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:26.154809    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:26.434231 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:26.434201    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"]
Sep 08 15:47:26.436016 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:47:26.435986    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a461b46_658e_49ba_ae2c_7a68668dbdd9.slice/crio-efa469fe1222514686495f5640693811137103a8ff07860f307306d7926534a2 WatchSource:0}: Error finding container efa469fe1222514686495f5640693811137103a8ff07860f307306d7926534a2: Status 404 returned error can't find the container with id efa469fe1222514686495f5640693811137103a8ff07860f307306d7926534a2
Sep 08 15:47:27.424635 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:27.424581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn" event={"ID":"1a461b46-658e-49ba-ae2c-7a68668dbdd9","Type":"ContainerStarted","Data":"89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e"}
Sep 08 15:47:27.425169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:27.424718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn" event={"ID":"1a461b46-658e-49ba-ae2c-7a68668dbdd9","Type":"ContainerStarted","Data":"efa469fe1222514686495f5640693811137103a8ff07860f307306d7926534a2"}
Sep 08 15:47:27.425463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:27.425433    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:27.442824 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:27.442767    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn" podStartSLOduration=2.442745491 podStartE2EDuration="2.442745491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:47:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:47:27.440082631 +0000 UTC m=+893.883274866" watchObservedRunningTime="2026-09-08 15:47:27.442745491 +0000 UTC m=+893.885937701"
Sep 08 15:47:38.432896 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:38.432860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:48.910443 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:48.910405    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"]
Sep 08 15:47:48.915463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:48.915445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:48.918041 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:48.917745    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:47:48.921580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:48.921554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"]
Sep 08 15:47:48.965841 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:48.965800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1990-4228-4bea-8e14-73c174f35aaa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv\" (UID: \"6dcd1990-4228-4bea-8e14-73c174f35aaa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:49.067371 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.067333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1990-4228-4bea-8e14-73c174f35aaa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv\" (UID: \"6dcd1990-4228-4bea-8e14-73c174f35aaa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:49.067966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.067948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1990-4228-4bea-8e14-73c174f35aaa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv\" (UID: \"6dcd1990-4228-4bea-8e14-73c174f35aaa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:49.228994 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.228958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:49.342801 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.342769    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"]
Sep 08 15:47:49.346727 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.346710    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:49.353364 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.353273    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"]
Sep 08 15:47:49.472268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.472233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e63e7d6-7417-4495-b246-fb894cf86368-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-36c51d-7lnw6\" (UID: \"2e63e7d6-7417-4495-b246-fb894cf86368\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:49.497718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.497690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"]
Sep 08 15:47:49.500114 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:47:49.500082    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcd1990_4228_4bea_8e14_73c174f35aaa.slice/crio-36227961f30094579a6be5d1d14bfe8f1cf3cfadaa5827f9e1465b4dea122b0c WatchSource:0}: Error finding container 36227961f30094579a6be5d1d14bfe8f1cf3cfadaa5827f9e1465b4dea122b0c: Status 404 returned error can't find the container with id 36227961f30094579a6be5d1d14bfe8f1cf3cfadaa5827f9e1465b4dea122b0c
Sep 08 15:47:49.556754 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.556720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv" event={"ID":"6dcd1990-4228-4bea-8e14-73c174f35aaa","Type":"ContainerStarted","Data":"36227961f30094579a6be5d1d14bfe8f1cf3cfadaa5827f9e1465b4dea122b0c"}
Sep 08 15:47:49.573389 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.573359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e63e7d6-7417-4495-b246-fb894cf86368-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-36c51d-7lnw6\" (UID: \"2e63e7d6-7417-4495-b246-fb894cf86368\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:49.574019 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.573996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e63e7d6-7417-4495-b246-fb894cf86368-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-36c51d-7lnw6\" (UID: \"2e63e7d6-7417-4495-b246-fb894cf86368\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:49.660067 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.660035    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:49.933830 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:49.933800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"]
Sep 08 15:47:49.935544 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:47:49.935497    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e63e7d6_7417_4495_b246_fb894cf86368.slice/crio-ed064abee53bc449a5ed9aca41626c4e3708dd02d4f1fa505e1957a900570cc1 WatchSource:0}: Error finding container ed064abee53bc449a5ed9aca41626c4e3708dd02d4f1fa505e1957a900570cc1: Status 404 returned error can't find the container with id ed064abee53bc449a5ed9aca41626c4e3708dd02d4f1fa505e1957a900570cc1
Sep 08 15:47:50.571523 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:50.571486    2563 generic.go:356] "Generic (PLEG): container finished" podID="6dcd1990-4228-4bea-8e14-73c174f35aaa" containerID="0829669cc0ad0c308f049ebac0f164a9e92dda3e714a82f86efd65449b6baf23" exitCode=0
Sep 08 15:47:50.571736 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:50.571570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv" event={"ID":"6dcd1990-4228-4bea-8e14-73c174f35aaa","Type":"ContainerDied","Data":"0829669cc0ad0c308f049ebac0f164a9e92dda3e714a82f86efd65449b6baf23"}
Sep 08 15:47:50.573478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:50.573454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6" event={"ID":"2e63e7d6-7417-4495-b246-fb894cf86368","Type":"ContainerStarted","Data":"9d8b1d0731f0d50ab7be45573dfd2052636b0a29439aee3b44ef44a08b0f516e"}
Sep 08 15:47:50.573600 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:50.573496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6" event={"ID":"2e63e7d6-7417-4495-b246-fb894cf86368","Type":"ContainerStarted","Data":"ed064abee53bc449a5ed9aca41626c4e3708dd02d4f1fa505e1957a900570cc1"}
Sep 08 15:47:50.601360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:50.601306    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6" podStartSLOduration=1.601275706 podStartE2EDuration="1.601275706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:47:50.599148591 +0000 UTC m=+917.042340810" watchObservedRunningTime="2026-09-08 15:47:50.601275706 +0000 UTC m=+917.044467917"
Sep 08 15:47:51.587387 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:51.587346    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e63e7d6-7417-4495-b246-fb894cf86368" containerID="9d8b1d0731f0d50ab7be45573dfd2052636b0a29439aee3b44ef44a08b0f516e" exitCode=0
Sep 08 15:47:51.587791 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:51.587433    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6" event={"ID":"2e63e7d6-7417-4495-b246-fb894cf86368","Type":"ContainerDied","Data":"9d8b1d0731f0d50ab7be45573dfd2052636b0a29439aee3b44ef44a08b0f516e"}
Sep 08 15:47:51.736865 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:51.736838    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:51.796516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:51.796475    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1990-4228-4bea-8e14-73c174f35aaa-maas-api-service-ca\") pod \"6dcd1990-4228-4bea-8e14-73c174f35aaa\" (UID: \"6dcd1990-4228-4bea-8e14-73c174f35aaa\") "
Sep 08 15:47:51.796898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:51.796872    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcd1990-4228-4bea-8e14-73c174f35aaa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6dcd1990-4228-4bea-8e14-73c174f35aaa" (UID: "6dcd1990-4228-4bea-8e14-73c174f35aaa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:51.897739 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:51.897650    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcd1990-4228-4bea-8e14-73c174f35aaa-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:52.595187 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.595155    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"
Sep 08 15:47:52.595615 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.595203    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv" event={"ID":"6dcd1990-4228-4bea-8e14-73c174f35aaa","Type":"ContainerDied","Data":"36227961f30094579a6be5d1d14bfe8f1cf3cfadaa5827f9e1465b4dea122b0c"}
Sep 08 15:47:52.595615 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.595251    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36227961f30094579a6be5d1d14bfe8f1cf3cfadaa5827f9e1465b4dea122b0c"
Sep 08 15:47:52.743604 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.743574    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:52.805982 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.805945    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e63e7d6-7417-4495-b246-fb894cf86368-maas-api-service-ca\") pod \"2e63e7d6-7417-4495-b246-fb894cf86368\" (UID: \"2e63e7d6-7417-4495-b246-fb894cf86368\") "
Sep 08 15:47:52.806368 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.806341    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e63e7d6-7417-4495-b246-fb894cf86368-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e63e7d6-7417-4495-b246-fb894cf86368" (UID: "2e63e7d6-7417-4495-b246-fb894cf86368"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:52.907251 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:52.907159    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e63e7d6-7417-4495-b246-fb894cf86368-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:53.602009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:53.601975    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"
Sep 08 15:47:53.602424 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:53.601981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6" event={"ID":"2e63e7d6-7417-4495-b246-fb894cf86368","Type":"ContainerDied","Data":"ed064abee53bc449a5ed9aca41626c4e3708dd02d4f1fa505e1957a900570cc1"}
Sep 08 15:47:53.602424 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:53.602084    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed064abee53bc449a5ed9aca41626c4e3708dd02d4f1fa505e1957a900570cc1"
Sep 08 15:47:55.933111 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:55.933078    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"]
Sep 08 15:47:55.933569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:55.933302    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn" podUID="1a461b46-658e-49ba-ae2c-7a68668dbdd9" containerName="payload-processing" containerID="cri-o://89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e" gracePeriod=30
Sep 08 15:47:56.084917 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.084880    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"]
Sep 08 15:47:56.088580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.088549    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-a82afc10-vp2wv"]
Sep 08 15:47:56.102821 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.102544    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dcd1990-4228-4bea-8e14-73c174f35aaa" path="/var/lib/kubelet/pods/6dcd1990-4228-4bea-8e14-73c174f35aaa/volumes"
Sep 08 15:47:56.184333 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.184248    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:56.342421 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.342384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6426\" (UniqueName: \"kubernetes.io/projected/1a461b46-658e-49ba-ae2c-7a68668dbdd9-kube-api-access-k6426\") pod \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") "
Sep 08 15:47:56.342608 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.342534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a461b46-658e-49ba-ae2c-7a68668dbdd9-plugins-config-volume\") pod \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\" (UID: \"1a461b46-658e-49ba-ae2c-7a68668dbdd9\") "
Sep 08 15:47:56.342983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.342931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a461b46-658e-49ba-ae2c-7a68668dbdd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a461b46-658e-49ba-ae2c-7a68668dbdd9" (UID: "1a461b46-658e-49ba-ae2c-7a68668dbdd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:47:56.345130 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.345092    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a461b46-658e-49ba-ae2c-7a68668dbdd9-kube-api-access-k6426" (OuterVolumeSpecName: "kube-api-access-k6426") pod "1a461b46-658e-49ba-ae2c-7a68668dbdd9" (UID: "1a461b46-658e-49ba-ae2c-7a68668dbdd9"). InnerVolumeSpecName "kube-api-access-k6426". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:47:56.443688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.443599    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6426\" (UniqueName: \"kubernetes.io/projected/1a461b46-658e-49ba-ae2c-7a68668dbdd9-kube-api-access-k6426\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:56.443688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.443629    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a461b46-658e-49ba-ae2c-7a68668dbdd9-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:47:56.620711 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.620675    2563 generic.go:356] "Generic (PLEG): container finished" podID="1a461b46-658e-49ba-ae2c-7a68668dbdd9" containerID="89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e" exitCode=0
Sep 08 15:47:56.620882 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.620734    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"
Sep 08 15:47:56.620882 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.620769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn" event={"ID":"1a461b46-658e-49ba-ae2c-7a68668dbdd9","Type":"ContainerDied","Data":"89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e"}
Sep 08 15:47:56.620882 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.620846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn" event={"ID":"1a461b46-658e-49ba-ae2c-7a68668dbdd9","Type":"ContainerDied","Data":"efa469fe1222514686495f5640693811137103a8ff07860f307306d7926534a2"}
Sep 08 15:47:56.620882 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.620868    2563 scope.go:117] "RemoveContainer" containerID="89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e"
Sep 08 15:47:56.629954 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.629936    2563 scope.go:117] "RemoveContainer" containerID="89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e"
Sep 08 15:47:56.630228 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:47:56.630209    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e\": container with ID starting with 89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e not found: ID does not exist" containerID="89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e"
Sep 08 15:47:56.630328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.630236    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e"} err="failed to get container status \"89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e\": rpc error: code = NotFound desc = could not find container \"89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e\": container with ID starting with 89d4ba4b8fcc3d39473501f1ea2b2ea8a103da66081132c76bc1fe57433f2d7e not found: ID does not exist"
Sep 08 15:47:56.641784 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.641754    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"]
Sep 08 15:47:56.646229 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:56.646201    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a82afc10-6c66bdf85c-brfcn"]
Sep 08 15:47:58.100785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:47:58.100751    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a461b46-658e-49ba-ae2c-7a68668dbdd9" path="/var/lib/kubelet/pods/1a461b46-658e-49ba-ae2c-7a68668dbdd9/volumes"
Sep 08 15:48:00.258297 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.258246    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"]
Sep 08 15:48:00.259055 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.259020    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5" podUID="f344958f-df88-4a01-a3e4-debc83a51bae" containerName="payload-pre-processing" containerID="cri-o://c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904" gracePeriod=30
Sep 08 15:48:00.478167 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.478123    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"]
Sep 08 15:48:00.480582 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.480563    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-36c51d-7lnw6"]
Sep 08 15:48:00.526120 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.526092    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:48:00.644251 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.644213    2563 generic.go:356] "Generic (PLEG): container finished" podID="f344958f-df88-4a01-a3e4-debc83a51bae" containerID="c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904" exitCode=0
Sep 08 15:48:00.644251 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.644255    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"
Sep 08 15:48:00.644503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.644309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5" event={"ID":"f344958f-df88-4a01-a3e4-debc83a51bae","Type":"ContainerDied","Data":"c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904"}
Sep 08 15:48:00.644503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.644361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5" event={"ID":"f344958f-df88-4a01-a3e4-debc83a51bae","Type":"ContainerDied","Data":"8ffcac83194eff9a9f74b65d65bf6f55e6eafb4376052ba63fd62e9cda6e61f5"}
Sep 08 15:48:00.644503 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.644385    2563 scope.go:117] "RemoveContainer" containerID="c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904"
Sep 08 15:48:00.653339 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.653318    2563 scope.go:117] "RemoveContainer" containerID="c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904"
Sep 08 15:48:00.653675 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:48:00.653656    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904\": container with ID starting with c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904 not found: ID does not exist" containerID="c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904"
Sep 08 15:48:00.653717 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.653686    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904"} err="failed to get container status \"c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904\": rpc error: code = NotFound desc = could not find container \"c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904\": container with ID starting with c2e57f2ddd816254b2875001ae54dc4cc2c8885e9b0df719388ec16f9ecd1904 not found: ID does not exist"
Sep 08 15:48:00.687638 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.687601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f344958f-df88-4a01-a3e4-debc83a51bae-plugins-config-volume\") pod \"f344958f-df88-4a01-a3e4-debc83a51bae\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") "
Sep 08 15:48:00.687779 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.687694    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm7nw\" (UniqueName: \"kubernetes.io/projected/f344958f-df88-4a01-a3e4-debc83a51bae-kube-api-access-rm7nw\") pod \"f344958f-df88-4a01-a3e4-debc83a51bae\" (UID: \"f344958f-df88-4a01-a3e4-debc83a51bae\") "
Sep 08 15:48:00.687996 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.687970    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f344958f-df88-4a01-a3e4-debc83a51bae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f344958f-df88-4a01-a3e4-debc83a51bae" (UID: "f344958f-df88-4a01-a3e4-debc83a51bae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:48:00.689836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.689814    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f344958f-df88-4a01-a3e4-debc83a51bae-kube-api-access-rm7nw" (OuterVolumeSpecName: "kube-api-access-rm7nw") pod "f344958f-df88-4a01-a3e4-debc83a51bae" (UID: "f344958f-df88-4a01-a3e4-debc83a51bae"). InnerVolumeSpecName "kube-api-access-rm7nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:48:00.788842 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.788752    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f344958f-df88-4a01-a3e4-debc83a51bae-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:00.788842 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.788784    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm7nw\" (UniqueName: \"kubernetes.io/projected/f344958f-df88-4a01-a3e4-debc83a51bae-kube-api-access-rm7nw\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:00.965953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.965918    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"]
Sep 08 15:48:00.970177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:00.970148    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-36c51d-7bbcd4dfdb-dbhh5"]
Sep 08 15:48:02.102089 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:02.102053    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e63e7d6-7417-4495-b246-fb894cf86368" path="/var/lib/kubelet/pods/2e63e7d6-7417-4495-b246-fb894cf86368/volumes"
Sep 08 15:48:02.102627 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:02.102445    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f344958f-df88-4a01-a3e4-debc83a51bae" path="/var/lib/kubelet/pods/f344958f-df88-4a01-a3e4-debc83a51bae/volumes"
Sep 08 15:48:40.841351 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841316    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"]
Sep 08 15:48:40.841891 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841872    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f344958f-df88-4a01-a3e4-debc83a51bae" containerName="payload-pre-processing"
Sep 08 15:48:40.841891 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841893    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f344958f-df88-4a01-a3e4-debc83a51bae" containerName="payload-pre-processing"
Sep 08 15:48:40.841986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841901    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a461b46-658e-49ba-ae2c-7a68668dbdd9" containerName="payload-processing"
Sep 08 15:48:40.841986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841906    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a461b46-658e-49ba-ae2c-7a68668dbdd9" containerName="payload-processing"
Sep 08 15:48:40.841986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841923    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dcd1990-4228-4bea-8e14-73c174f35aaa" containerName="revoke-keys"
Sep 08 15:48:40.841986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841929    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dcd1990-4228-4bea-8e14-73c174f35aaa" containerName="revoke-keys"
Sep 08 15:48:40.841986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841941    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e63e7d6-7417-4495-b246-fb894cf86368" containerName="revoke-keys"
Sep 08 15:48:40.841986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.841946    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e63e7d6-7417-4495-b246-fb894cf86368" containerName="revoke-keys"
Sep 08 15:48:40.842166 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.842008    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e63e7d6-7417-4495-b246-fb894cf86368" containerName="revoke-keys"
Sep 08 15:48:40.842166 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.842017    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a461b46-658e-49ba-ae2c-7a68668dbdd9" containerName="payload-processing"
Sep 08 15:48:40.842166 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.842024    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dcd1990-4228-4bea-8e14-73c174f35aaa" containerName="revoke-keys"
Sep 08 15:48:40.842166 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.842033    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f344958f-df88-4a01-a3e4-debc83a51bae" containerName="payload-pre-processing"
Sep 08 15:48:40.846721 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.846701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:40.849134 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.849109    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a057c5\"/\"openshift-service-ca.crt\""
Sep 08 15:48:40.849271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.849198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a057c5\"/\"kube-root-ca.crt\""
Sep 08 15:48:40.850058 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.850036    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a057c5\"/\"default-dockercfg-rr7dv\""
Sep 08 15:48:40.850255 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.850074    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a057c5\"/\"test-model-a057c5-kserve-self-signed-certs\""
Sep 08 15:48:40.853295 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.853255    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"]
Sep 08 15:48:40.975220 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.975186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-home\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:40.975428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.975229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-tmp-dir\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:40.975428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.975254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62bf0655-f0c3-4307-861c-e4f95b33511d-tls-certs\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:40.975428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.975352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-dshm\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:40.975428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.975374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-model-cache\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:40.975665 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:40.975455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xh7n\" (UniqueName: \"kubernetes.io/projected/62bf0655-f0c3-4307-861c-e4f95b33511d-kube-api-access-4xh7n\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076060 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076018    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-home\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-tmp-dir\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62bf0655-f0c3-4307-861c-e4f95b33511d-tls-certs\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-dshm\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-model-cache\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076256 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xh7n\" (UniqueName: \"kubernetes.io/projected/62bf0655-f0c3-4307-861c-e4f95b33511d-kube-api-access-4xh7n\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076557 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-home\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076557 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-tmp-dir\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.076651 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.076583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-model-cache\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.078425 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.078403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-dshm\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.078652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.078634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62bf0655-f0c3-4307-861c-e4f95b33511d-tls-certs\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.083744 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.083720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xh7n\" (UniqueName: \"kubernetes.io/projected/62bf0655-f0c3-4307-861c-e4f95b33511d-kube-api-access-4xh7n\") pod \"test-model-a057c5-kserve-596c98d8c-8p88k\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") " pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.156920 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.156838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:41.448907 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.448868    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"]
Sep 08 15:48:41.449996 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:48:41.449967    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62bf0655_f0c3_4307_861c_e4f95b33511d.slice/crio-33c720e56e08f1b875a8472974ffd9ac24be4c07145eb1a92ea4536e9c48b915 WatchSource:0}: Error finding container 33c720e56e08f1b875a8472974ffd9ac24be4c07145eb1a92ea4536e9c48b915: Status 404 returned error can't find the container with id 33c720e56e08f1b875a8472974ffd9ac24be4c07145eb1a92ea4536e9c48b915
Sep 08 15:48:41.870241 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.870142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k" event={"ID":"62bf0655-f0c3-4307-861c-e4f95b33511d","Type":"ContainerStarted","Data":"7c1ff33c6837c8b80ba032cf4df50cf384cb5a654909f1f7837c7829f13c975a"}
Sep 08 15:48:41.870241 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.870190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k" event={"ID":"62bf0655-f0c3-4307-861c-e4f95b33511d","Type":"ContainerStarted","Data":"33c720e56e08f1b875a8472974ffd9ac24be4c07145eb1a92ea4536e9c48b915"}
Sep 08 15:48:41.889946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:41.889894    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k" podStartSLOduration=1.889878071 podStartE2EDuration="1.889878071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:48:41.888208024 +0000 UTC m=+968.331400255" watchObservedRunningTime="2026-09-08 15:48:41.889878071 +0000 UTC m=+968.333070279"
Sep 08 15:48:43.687297 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:43.687251    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"]
Sep 08 15:48:43.687825 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:43.687470    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" containerID="cri-o://a8a7cfbdf7cb01872110fd06821ab4c2206c63249ccf21fd7426a35806d0a7a4" gracePeriod=30
Sep 08 15:48:48.224340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:48.224304    2563 patch_prober.go:28] interesting pod/e2e-autores-36c51d-openshift-default-589558d68b-njsdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 15:48:48.224746 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:48.224363    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 15:48:48.914127 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:48.914092    2563 generic.go:356] "Generic (PLEG): container finished" podID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerID="a8a7cfbdf7cb01872110fd06821ab4c2206c63249ccf21fd7426a35806d0a7a4" exitCode=0
Sep 08 15:48:48.914341 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:48.914121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" event={"ID":"ffe587bb-ca16-4a35-b7f5-98f546c68d64","Type":"ContainerDied","Data":"a8a7cfbdf7cb01872110fd06821ab4c2206c63249ccf21fd7426a35806d0a7a4"}
Sep 08 15:48:48.914341 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:48.914197    2563 scope.go:117] "RemoveContainer" containerID="f4ac41293262fcffdfe28224481c5250c19b7d69b088af6513a0a26e05bc3030"
Sep 08 15:48:48.955194 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:48.955132    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:48:49.048768 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.048735    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"]
Sep 08 15:48:49.049193 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.049181    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy"
Sep 08 15:48:49.049235 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.049195    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy"
Sep 08 15:48:49.049235 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.049208    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy"
Sep 08 15:48:49.049235 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.049214    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy"
Sep 08 15:48:49.049345 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.049274    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy"
Sep 08 15:48:49.049345 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.049282    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" containerName="istio-proxy"
Sep 08 15:48:49.052493 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.052470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.057525 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.057502    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-bb8457-dockercfg-cg69x\""
Sep 08 15:48:49.057748 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.057724    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-bb8457\""
Sep 08 15:48:49.062665 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062644    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-certs\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.062795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062725    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-socket\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.062795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062745    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwhhd\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-kube-api-access-qwhhd\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.062795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062788    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-credential-socket\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.062958 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-podinfo\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.062958 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-envoy\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.063055 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:48:49.063055 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.062977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-data\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.063055 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063050    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-token\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.063210 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063078    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istiod-ca-cert\") pod \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\" (UID: \"ffe587bb-ca16-4a35-b7f5-98f546c68d64\") "
Sep 08 15:48:49.063210 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:48:49.063210 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063138    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:48:49.063373 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063245    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-data" (OuterVolumeSpecName: "istio-data") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:48:49.063540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063521    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.063617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063559    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.063617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063577    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.063617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063591    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.063617 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.063519    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:48:49.065192 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.065167    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-kube-api-access-qwhhd" (OuterVolumeSpecName: "kube-api-access-qwhhd") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "kube-api-access-qwhhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:48:49.065279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.065187    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-token" (OuterVolumeSpecName: "istio-token") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:48:49.065497 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.065463    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:48:49.065497 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.065464    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ffe587bb-ca16-4a35-b7f5-98f546c68d64" (UID: "ffe587bb-ca16-4a35-b7f5-98f546c68d64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:48:49.069524 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.069497    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"]
Sep 08 15:48:49.164687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c3ba7b-ba25-430a-ac71-c6019836df5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.164880 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm5xj\" (UniqueName: \"kubernetes.io/projected/d4c3ba7b-ba25-430a-ac71-c6019836df5a-kube-api-access-cm5xj\") pod \"payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.164932 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164880    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.164932 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164904    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.164932 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164914    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.164932 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164921    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffe587bb-ca16-4a35-b7f5-98f546c68d64-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.164932 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.164930    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwhhd\" (UniqueName: \"kubernetes.io/projected/ffe587bb-ca16-4a35-b7f5-98f546c68d64-kube-api-access-qwhhd\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:48:49.266092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.265999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c3ba7b-ba25-430a-ac71-c6019836df5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.266092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.266054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm5xj\" (UniqueName: \"kubernetes.io/projected/d4c3ba7b-ba25-430a-ac71-c6019836df5a-kube-api-access-cm5xj\") pod \"payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.266760 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.266740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c3ba7b-ba25-430a-ac71-c6019836df5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.276254 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.276233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm5xj\" (UniqueName: \"kubernetes.io/projected/d4c3ba7b-ba25-430a-ac71-c6019836df5a-kube-api-access-cm5xj\") pod \"payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.363107 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.363066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.640675 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.640641    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"]
Sep 08 15:48:49.642733 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:48:49.642706    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c3ba7b_ba25_430a_ac71_c6019836df5a.slice/crio-035aa68a884ec4fa4745416e9730c0e23678b24ec6309672c95fe821a306871a WatchSource:0}: Error finding container 035aa68a884ec4fa4745416e9730c0e23678b24ec6309672c95fe821a306871a: Status 404 returned error can't find the container with id 035aa68a884ec4fa4745416e9730c0e23678b24ec6309672c95fe821a306871a
Sep 08 15:48:49.920856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.920758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr" event={"ID":"ffe587bb-ca16-4a35-b7f5-98f546c68d64","Type":"ContainerDied","Data":"0e7203b132027a2daadaca3262610a2e5fa6d2fdb667ddef3a9a41eb2b61420b"}
Sep 08 15:48:49.920856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.920820    2563 scope.go:117] "RemoveContainer" containerID="a8a7cfbdf7cb01872110fd06821ab4c2206c63249ccf21fd7426a35806d0a7a4"
Sep 08 15:48:49.920856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.920772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"
Sep 08 15:48:49.922653 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.922626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr" event={"ID":"d4c3ba7b-ba25-430a-ac71-c6019836df5a","Type":"ContainerStarted","Data":"ab746193bf7a1cc1bf08169fa805e3c3746cbef668fab6beabb5645ad6ef9600"}
Sep 08 15:48:49.922749 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.922682    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:48:49.922749 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.922699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr" event={"ID":"d4c3ba7b-ba25-430a-ac71-c6019836df5a","Type":"ContainerStarted","Data":"035aa68a884ec4fa4745416e9730c0e23678b24ec6309672c95fe821a306871a"}
Sep 08 15:48:49.943526 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.943475    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr" podStartSLOduration=0.943459024 podStartE2EDuration="943.459024ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:48:49.941939775 +0000 UTC m=+976.385132022" watchObservedRunningTime="2026-09-08 15:48:49.943459024 +0000 UTC m=+976.386651244"
Sep 08 15:48:49.956876 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.956843    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"]
Sep 08 15:48:49.961044 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:49.961019    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-36c51d-openshift-default-589558d68b-njsdr"]
Sep 08 15:48:50.102279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:50.102247    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffe587bb-ca16-4a35-b7f5-98f546c68d64" path="/var/lib/kubelet/pods/ffe587bb-ca16-4a35-b7f5-98f546c68d64/volumes"
Sep 08 15:48:51.157043 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:51.157007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:51.157484 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:51.157059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:51.169964 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:51.169936    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:48:51.945731 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:48:51.945703    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:49:00.931496 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:00.931415    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:49:30.255965 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.255926    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"]
Sep 08 15:49:30.260724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.260706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:30.263015 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.262981    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:49:30.267084 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.267048    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"]
Sep 08 15:49:30.292048 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.292007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7\" (UID: \"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:30.393054 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.393015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7\" (UID: \"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:30.393699 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.393676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7\" (UID: \"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:30.573035 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.572942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:30.845399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:30.845309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"]
Sep 08 15:49:30.848751 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:49:30.848719    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebe84b1_c152_4a9e_af0e_1b2e9ee81609.slice/crio-e35f4d897190f3baa8c8081fc823a595c7a2f5ae10d13a42c09c4469c07ed877 WatchSource:0}: Error finding container e35f4d897190f3baa8c8081fc823a595c7a2f5ae10d13a42c09c4469c07ed877: Status 404 returned error can't find the container with id e35f4d897190f3baa8c8081fc823a595c7a2f5ae10d13a42c09c4469c07ed877
Sep 08 15:49:31.151544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:31.151439    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7" event={"ID":"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609","Type":"ContainerStarted","Data":"c4a34aa86bcf067c94eacf190bf9dd8d083ad1d2ca181178e05ca654224b1ba5"}
Sep 08 15:49:31.151544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:31.151499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7" event={"ID":"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609","Type":"ContainerStarted","Data":"e35f4d897190f3baa8c8081fc823a595c7a2f5ae10d13a42c09c4469c07ed877"}
Sep 08 15:49:31.168169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:31.168111    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7" podStartSLOduration=1.168093031 podStartE2EDuration="1.168093031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:31.165630194 +0000 UTC m=+1017.608822421" watchObservedRunningTime="2026-09-08 15:49:31.168093031 +0000 UTC m=+1017.611285296"
Sep 08 15:49:32.158686 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:32.158649    2563 generic.go:356] "Generic (PLEG): container finished" podID="0ebe84b1-c152-4a9e-af0e-1b2e9ee81609" containerID="c4a34aa86bcf067c94eacf190bf9dd8d083ad1d2ca181178e05ca654224b1ba5" exitCode=0
Sep 08 15:49:32.159136 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:32.158703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7" event={"ID":"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609","Type":"ContainerDied","Data":"c4a34aa86bcf067c94eacf190bf9dd8d083ad1d2ca181178e05ca654224b1ba5"}
Sep 08 15:49:33.314761 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:33.314736    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:33.424963 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:33.424935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609-maas-api-service-ca\") pod \"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609\" (UID: \"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609\") "
Sep 08 15:49:33.425352 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:33.425325    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ebe84b1-c152-4a9e-af0e-1b2e9ee81609" (UID: "0ebe84b1-c152-4a9e-af0e-1b2e9ee81609"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:49:33.526335 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:33.526274    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:49:34.171871 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:34.171844    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"
Sep 08 15:49:34.171871 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:34.171859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7" event={"ID":"0ebe84b1-c152-4a9e-af0e-1b2e9ee81609","Type":"ContainerDied","Data":"e35f4d897190f3baa8c8081fc823a595c7a2f5ae10d13a42c09c4469c07ed877"}
Sep 08 15:49:34.172049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:34.171904    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e35f4d897190f3baa8c8081fc823a595c7a2f5ae10d13a42c09c4469c07ed877"
Sep 08 15:49:37.430469 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.430435    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"]
Sep 08 15:49:37.430871 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.430859    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ebe84b1-c152-4a9e-af0e-1b2e9ee81609" containerName="revoke-keys"
Sep 08 15:49:37.430871 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.430873    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ebe84b1-c152-4a9e-af0e-1b2e9ee81609" containerName="revoke-keys"
Sep 08 15:49:37.430953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.430944    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ebe84b1-c152-4a9e-af0e-1b2e9ee81609" containerName="revoke-keys"
Sep 08 15:49:37.435348 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.435331    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.437798 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.437755    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bbf4a143-dockercfg-8m6cf\""
Sep 08 15:49:37.437923 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.437811    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bbf4a143\""
Sep 08 15:49:37.441650 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.441627    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"]
Sep 08 15:49:37.565966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.565933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h2wp\" (UniqueName: \"kubernetes.io/projected/3d1947d5-132e-4624-822e-67f558295e39-kube-api-access-7h2wp\") pod \"payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") " pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.565966 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.565968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1947d5-132e-4624-822e-67f558295e39-plugins-config-volume\") pod \"payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") " pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.666795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.666751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h2wp\" (UniqueName: \"kubernetes.io/projected/3d1947d5-132e-4624-822e-67f558295e39-kube-api-access-7h2wp\") pod \"payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") " pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.666795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.666794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1947d5-132e-4624-822e-67f558295e39-plugins-config-volume\") pod \"payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") " pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.667526 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.667508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1947d5-132e-4624-822e-67f558295e39-plugins-config-volume\") pod \"payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") " pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.674862 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.674837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h2wp\" (UniqueName: \"kubernetes.io/projected/3d1947d5-132e-4624-822e-67f558295e39-kube-api-access-7h2wp\") pod \"payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") " pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:37.747968 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:37.747878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:38.022341 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:38.022313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"]
Sep 08 15:49:38.024281 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:49:38.024255    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d1947d5_132e_4624_822e_67f558295e39.slice/crio-46e6ce2918889f0be38025d80074ff158fe80ff2b09005cf654c2765b0eae2c9 WatchSource:0}: Error finding container 46e6ce2918889f0be38025d80074ff158fe80ff2b09005cf654c2765b0eae2c9: Status 404 returned error can't find the container with id 46e6ce2918889f0be38025d80074ff158fe80ff2b09005cf654c2765b0eae2c9
Sep 08 15:49:38.197340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:38.197300    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw" event={"ID":"3d1947d5-132e-4624-822e-67f558295e39","Type":"ContainerStarted","Data":"97d779d8cc3a5bfc8e95b790378ec29eb0229322d75987edf44a3243a7346e74"}
Sep 08 15:49:38.197528 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:38.197351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw" event={"ID":"3d1947d5-132e-4624-822e-67f558295e39","Type":"ContainerStarted","Data":"46e6ce2918889f0be38025d80074ff158fe80ff2b09005cf654c2765b0eae2c9"}
Sep 08 15:49:38.197528 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:38.197420    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:49:38.213568 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:38.213511    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw" podStartSLOduration=1.213493376 podStartE2EDuration="1.213493376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:49:38.211952047 +0000 UTC m=+1024.655144269" watchObservedRunningTime="2026-09-08 15:49:38.213493376 +0000 UTC m=+1024.656685584"
Sep 08 15:49:41.049705 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.049666    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"]
Sep 08 15:49:41.050385 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.050326    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr" podUID="d4c3ba7b-ba25-430a-ac71-c6019836df5a" containerName="payload-pre-processing" containerID="cri-o://ab746193bf7a1cc1bf08169fa805e3c3746cbef668fab6beabb5645ad6ef9600" gracePeriod=30
Sep 08 15:49:41.203226 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.203181    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"]
Sep 08 15:49:41.205659 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.205629    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-bb8457-t8bs7"]
Sep 08 15:49:41.216590 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.216549    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4c3ba7b-ba25-430a-ac71-c6019836df5a" containerID="ab746193bf7a1cc1bf08169fa805e3c3746cbef668fab6beabb5645ad6ef9600" exitCode=0
Sep 08 15:49:41.216769 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.216754    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr" event={"ID":"d4c3ba7b-ba25-430a-ac71-c6019836df5a","Type":"ContainerDied","Data":"ab746193bf7a1cc1bf08169fa805e3c3746cbef668fab6beabb5645ad6ef9600"}
Sep 08 15:49:41.328657 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.328633    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:49:41.403190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.403154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm5xj\" (UniqueName: \"kubernetes.io/projected/d4c3ba7b-ba25-430a-ac71-c6019836df5a-kube-api-access-cm5xj\") pod \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") "
Sep 08 15:49:41.403368 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.403244    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c3ba7b-ba25-430a-ac71-c6019836df5a-plugins-config-volume\") pod \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\" (UID: \"d4c3ba7b-ba25-430a-ac71-c6019836df5a\") "
Sep 08 15:49:41.403586 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.403567    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c3ba7b-ba25-430a-ac71-c6019836df5a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4c3ba7b-ba25-430a-ac71-c6019836df5a" (UID: "d4c3ba7b-ba25-430a-ac71-c6019836df5a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:49:41.403672 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.403655    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4c3ba7b-ba25-430a-ac71-c6019836df5a-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:49:41.405342 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.405316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c3ba7b-ba25-430a-ac71-c6019836df5a-kube-api-access-cm5xj" (OuterVolumeSpecName: "kube-api-access-cm5xj") pod "d4c3ba7b-ba25-430a-ac71-c6019836df5a" (UID: "d4c3ba7b-ba25-430a-ac71-c6019836df5a"). InnerVolumeSpecName "kube-api-access-cm5xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:49:41.504546 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:41.504507    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm5xj\" (UniqueName: \"kubernetes.io/projected/d4c3ba7b-ba25-430a-ac71-c6019836df5a-kube-api-access-cm5xj\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:49:42.102580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:42.102538    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ebe84b1-c152-4a9e-af0e-1b2e9ee81609" path="/var/lib/kubelet/pods/0ebe84b1-c152-4a9e-af0e-1b2e9ee81609/volumes"
Sep 08 15:49:42.225453 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:42.225421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr" event={"ID":"d4c3ba7b-ba25-430a-ac71-c6019836df5a","Type":"ContainerDied","Data":"035aa68a884ec4fa4745416e9730c0e23678b24ec6309672c95fe821a306871a"}
Sep 08 15:49:42.225637 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:42.225487    2563 scope.go:117] "RemoveContainer" containerID="ab746193bf7a1cc1bf08169fa805e3c3746cbef668fab6beabb5645ad6ef9600"
Sep 08 15:49:42.225637 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:42.225429    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"
Sep 08 15:49:42.244946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:42.244917    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"]
Sep 08 15:49:42.249998 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:42.249971    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bb8457-7b55598478-fmzmr"]
Sep 08 15:49:44.101111 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:44.101076    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c3ba7b-ba25-430a-ac71-c6019836df5a" path="/var/lib/kubelet/pods/d4c3ba7b-ba25-430a-ac71-c6019836df5a/volumes"
Sep 08 15:49:49.204678 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:49:49.204636    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:50:07.717440 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.717399    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"]
Sep 08 15:50:07.717894 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.717880    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c3ba7b-ba25-430a-ac71-c6019836df5a" containerName="payload-pre-processing"
Sep 08 15:50:07.717948 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.717899    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c3ba7b-ba25-430a-ac71-c6019836df5a" containerName="payload-pre-processing"
Sep 08 15:50:07.717997 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.717987    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c3ba7b-ba25-430a-ac71-c6019836df5a" containerName="payload-pre-processing"
Sep 08 15:50:07.724357 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.724319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:07.727477 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.727448    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:50:07.730546 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.730517    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"]
Sep 08 15:50:07.757463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.757419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/317b5276-f87e-4a29-9724-78839572c273-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb\" (UID: \"317b5276-f87e-4a29-9724-78839572c273\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:07.858372 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.858331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/317b5276-f87e-4a29-9724-78839572c273-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb\" (UID: \"317b5276-f87e-4a29-9724-78839572c273\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:07.859011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:07.858981    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/317b5276-f87e-4a29-9724-78839572c273-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb\" (UID: \"317b5276-f87e-4a29-9724-78839572c273\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:08.038650 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:08.038556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:08.310229 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:08.310199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"]
Sep 08 15:50:08.312634 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:50:08.312605    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317b5276_f87e_4a29_9724_78839572c273.slice/crio-1d0be8e291692a9f3a8128addb42f6947eb6973c00aeb00b5cf7efdef5958e48 WatchSource:0}: Error finding container 1d0be8e291692a9f3a8128addb42f6947eb6973c00aeb00b5cf7efdef5958e48: Status 404 returned error can't find the container with id 1d0be8e291692a9f3a8128addb42f6947eb6973c00aeb00b5cf7efdef5958e48
Sep 08 15:50:08.381467 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:08.381432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb" event={"ID":"317b5276-f87e-4a29-9724-78839572c273","Type":"ContainerStarted","Data":"1d0be8e291692a9f3a8128addb42f6947eb6973c00aeb00b5cf7efdef5958e48"}
Sep 08 15:50:09.388836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:09.388739    2563 generic.go:356] "Generic (PLEG): container finished" podID="317b5276-f87e-4a29-9724-78839572c273" containerID="5fb4d0991048acc52a0af611100b61d8a32193827492a1bec3a562935d55b5fb" exitCode=0
Sep 08 15:50:09.389220 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:09.388829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb" event={"ID":"317b5276-f87e-4a29-9724-78839572c273","Type":"ContainerDied","Data":"5fb4d0991048acc52a0af611100b61d8a32193827492a1bec3a562935d55b5fb"}
Sep 08 15:50:10.529778 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:10.529752    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:10.585735 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:10.585699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/317b5276-f87e-4a29-9724-78839572c273-maas-api-service-ca\") pod \"317b5276-f87e-4a29-9724-78839572c273\" (UID: \"317b5276-f87e-4a29-9724-78839572c273\") "
Sep 08 15:50:10.586055 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:10.586033    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/317b5276-f87e-4a29-9724-78839572c273-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "317b5276-f87e-4a29-9724-78839572c273" (UID: "317b5276-f87e-4a29-9724-78839572c273"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:50:10.687002 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:10.686963    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/317b5276-f87e-4a29-9724-78839572c273-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:11.401034 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:11.400999    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"
Sep 08 15:50:11.401034 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:11.401030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb" event={"ID":"317b5276-f87e-4a29-9724-78839572c273","Type":"ContainerDied","Data":"1d0be8e291692a9f3a8128addb42f6947eb6973c00aeb00b5cf7efdef5958e48"}
Sep 08 15:50:11.401260 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:11.401074    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d0be8e291692a9f3a8128addb42f6947eb6973c00aeb00b5cf7efdef5958e48"
Sep 08 15:50:18.246850 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.246818    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"]
Sep 08 15:50:18.247360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.247032    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw" podUID="3d1947d5-132e-4624-822e-67f558295e39" containerName="payload-processing" containerID="cri-o://97d779d8cc3a5bfc8e95b790378ec29eb0229322d75987edf44a3243a7346e74" gracePeriod=30
Sep 08 15:50:18.414381 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.414336    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"]
Sep 08 15:50:18.416274 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.416252    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bbf4a143-d5rqb"]
Sep 08 15:50:18.443233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.443200    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d1947d5-132e-4624-822e-67f558295e39" containerID="97d779d8cc3a5bfc8e95b790378ec29eb0229322d75987edf44a3243a7346e74" exitCode=0
Sep 08 15:50:18.443426 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.443275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw" event={"ID":"3d1947d5-132e-4624-822e-67f558295e39","Type":"ContainerDied","Data":"97d779d8cc3a5bfc8e95b790378ec29eb0229322d75987edf44a3243a7346e74"}
Sep 08 15:50:18.510718 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.510659    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:50:18.567364 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.567326    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h2wp\" (UniqueName: \"kubernetes.io/projected/3d1947d5-132e-4624-822e-67f558295e39-kube-api-access-7h2wp\") pod \"3d1947d5-132e-4624-822e-67f558295e39\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") "
Sep 08 15:50:18.567640 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.567619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1947d5-132e-4624-822e-67f558295e39-plugins-config-volume\") pod \"3d1947d5-132e-4624-822e-67f558295e39\" (UID: \"3d1947d5-132e-4624-822e-67f558295e39\") "
Sep 08 15:50:18.568088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.568058    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d1947d5-132e-4624-822e-67f558295e39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d1947d5-132e-4624-822e-67f558295e39" (UID: "3d1947d5-132e-4624-822e-67f558295e39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:50:18.570127 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.570098    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d1947d5-132e-4624-822e-67f558295e39-kube-api-access-7h2wp" (OuterVolumeSpecName: "kube-api-access-7h2wp") pod "3d1947d5-132e-4624-822e-67f558295e39" (UID: "3d1947d5-132e-4624-822e-67f558295e39"). InnerVolumeSpecName "kube-api-access-7h2wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:50:18.669955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.669226    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h2wp\" (UniqueName: \"kubernetes.io/projected/3d1947d5-132e-4624-822e-67f558295e39-kube-api-access-7h2wp\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:18.669955 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:18.669257    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d1947d5-132e-4624-822e-67f558295e39-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:19.451157 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:19.451129    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"
Sep 08 15:50:19.451157 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:19.451134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw" event={"ID":"3d1947d5-132e-4624-822e-67f558295e39","Type":"ContainerDied","Data":"46e6ce2918889f0be38025d80074ff158fe80ff2b09005cf654c2765b0eae2c9"}
Sep 08 15:50:19.451701 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:19.451197    2563 scope.go:117] "RemoveContainer" containerID="97d779d8cc3a5bfc8e95b790378ec29eb0229322d75987edf44a3243a7346e74"
Sep 08 15:50:19.472697 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:19.472664    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"]
Sep 08 15:50:19.477279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:19.477253    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bbf4a143-5bd8ffd4c7-cg9kw"]
Sep 08 15:50:20.101884 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:20.101848    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="317b5276-f87e-4a29-9724-78839572c273" path="/var/lib/kubelet/pods/317b5276-f87e-4a29-9724-78839572c273/volumes"
Sep 08 15:50:20.102249 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:20.102236    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d1947d5-132e-4624-822e-67f558295e39" path="/var/lib/kubelet/pods/3d1947d5-132e-4624-822e-67f558295e39/volumes"
Sep 08 15:50:37.463347 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.463312    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"]
Sep 08 15:50:37.604782 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.604746    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"]
Sep 08 15:50:37.605181 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.605169    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="317b5276-f87e-4a29-9724-78839572c273" containerName="revoke-keys"
Sep 08 15:50:37.605226 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.605183    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="317b5276-f87e-4a29-9724-78839572c273" containerName="revoke-keys"
Sep 08 15:50:37.605226 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.605207    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d1947d5-132e-4624-822e-67f558295e39" containerName="payload-processing"
Sep 08 15:50:37.605226 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.605212    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d1947d5-132e-4624-822e-67f558295e39" containerName="payload-processing"
Sep 08 15:50:37.605353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.605280    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d1947d5-132e-4624-822e-67f558295e39" containerName="payload-processing"
Sep 08 15:50:37.605353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.605309    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="317b5276-f87e-4a29-9724-78839572c273" containerName="revoke-keys"
Sep 08 15:50:37.608057 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.608037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:37.610345 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.610318    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:50:37.620331 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.620247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"]
Sep 08 15:50:37.772700 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.772614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabf600d-5583-40e8-a2f4-f8565ae617f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-28e012-d75l7\" (UID: \"dabf600d-5583-40e8-a2f4-f8565ae617f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:37.874282 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.874237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabf600d-5583-40e8-a2f4-f8565ae617f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-28e012-d75l7\" (UID: \"dabf600d-5583-40e8-a2f4-f8565ae617f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:37.874933 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.874910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabf600d-5583-40e8-a2f4-f8565ae617f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-28e012-d75l7\" (UID: \"dabf600d-5583-40e8-a2f4-f8565ae617f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:37.927418 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:37.927374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:38.200242 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:38.200215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"]
Sep 08 15:50:38.202255 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:50:38.202230    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddabf600d_5583_40e8_a2f4_f8565ae617f3.slice/crio-52b0970fc1d57be6062971e7416f158708282ea22f6610009f1dc668b4bc00ba WatchSource:0}: Error finding container 52b0970fc1d57be6062971e7416f158708282ea22f6610009f1dc668b4bc00ba: Status 404 returned error can't find the container with id 52b0970fc1d57be6062971e7416f158708282ea22f6610009f1dc668b4bc00ba
Sep 08 15:50:38.557245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:38.557204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7" event={"ID":"dabf600d-5583-40e8-a2f4-f8565ae617f3","Type":"ContainerStarted","Data":"eb65b3166388ea862689288d0c7fd3bd8a5d05a1e571eeb8c3febb789f9b1a0c"}
Sep 08 15:50:38.557656 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:38.557275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7" event={"ID":"dabf600d-5583-40e8-a2f4-f8565ae617f3","Type":"ContainerStarted","Data":"52b0970fc1d57be6062971e7416f158708282ea22f6610009f1dc668b4bc00ba"}
Sep 08 15:50:38.573489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:38.573441    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7" podStartSLOduration=1.5734260770000001 podStartE2EDuration="1.573426077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:50:38.571126832 +0000 UTC m=+1085.014319078" watchObservedRunningTime="2026-09-08 15:50:38.573426077 +0000 UTC m=+1085.016618286"
Sep 08 15:50:39.564317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:39.564265    2563 generic.go:356] "Generic (PLEG): container finished" podID="dabf600d-5583-40e8-a2f4-f8565ae617f3" containerID="eb65b3166388ea862689288d0c7fd3bd8a5d05a1e571eeb8c3febb789f9b1a0c" exitCode=0
Sep 08 15:50:39.564764 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:39.564353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7" event={"ID":"dabf600d-5583-40e8-a2f4-f8565ae617f3","Type":"ContainerDied","Data":"eb65b3166388ea862689288d0c7fd3bd8a5d05a1e571eeb8c3febb789f9b1a0c"}
Sep 08 15:50:40.709501 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:40.709477    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:40.804502 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:40.804467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabf600d-5583-40e8-a2f4-f8565ae617f3-maas-api-service-ca\") pod \"dabf600d-5583-40e8-a2f4-f8565ae617f3\" (UID: \"dabf600d-5583-40e8-a2f4-f8565ae617f3\") "
Sep 08 15:50:40.804821 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:40.804799    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dabf600d-5583-40e8-a2f4-f8565ae617f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dabf600d-5583-40e8-a2f4-f8565ae617f3" (UID: "dabf600d-5583-40e8-a2f4-f8565ae617f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:50:40.905816 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:40.905717    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabf600d-5583-40e8-a2f4-f8565ae617f3-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:41.577168 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:41.577129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7" event={"ID":"dabf600d-5583-40e8-a2f4-f8565ae617f3","Type":"ContainerDied","Data":"52b0970fc1d57be6062971e7416f158708282ea22f6610009f1dc668b4bc00ba"}
Sep 08 15:50:41.577168 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:41.577154    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"
Sep 08 15:50:41.577410 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:41.577188    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52b0970fc1d57be6062971e7416f158708282ea22f6610009f1dc668b4bc00ba"
Sep 08 15:50:48.562019 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.561978    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"]
Sep 08 15:50:48.562555 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.562208    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8" podUID="c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" containerName="payload-processing" containerID="cri-o://25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b" gracePeriod=30
Sep 08 15:50:48.589711 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.589662    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"]
Sep 08 15:50:48.590685 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.590018    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz" podUID="d0cf30cd-6608-499b-9f91-a501080e5fae" containerName="payload-pre-processing" containerID="cri-o://2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8" gracePeriod=30
Sep 08 15:50:48.824620 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.824461    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"]
Sep 08 15:50:48.827621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.827594    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-28e012-d75l7"]
Sep 08 15:50:48.867776 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.867752    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:50:48.870778 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.870757    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:50:48.992458 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.992420    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2nf5\" (UniqueName: \"kubernetes.io/projected/d0cf30cd-6608-499b-9f91-a501080e5fae-kube-api-access-s2nf5\") pod \"d0cf30cd-6608-499b-9f91-a501080e5fae\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") "
Sep 08 15:50:48.992659 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.992475    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0cf30cd-6608-499b-9f91-a501080e5fae-plugins-config-volume\") pod \"d0cf30cd-6608-499b-9f91-a501080e5fae\" (UID: \"d0cf30cd-6608-499b-9f91-a501080e5fae\") "
Sep 08 15:50:48.992659 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.992604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6xt6\" (UniqueName: \"kubernetes.io/projected/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-kube-api-access-r6xt6\") pod \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") "
Sep 08 15:50:48.992781 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.992657    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-plugins-config-volume\") pod \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\" (UID: \"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa\") "
Sep 08 15:50:48.993020 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.992990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0cf30cd-6608-499b-9f91-a501080e5fae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0cf30cd-6608-499b-9f91-a501080e5fae" (UID: "d0cf30cd-6608-499b-9f91-a501080e5fae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:50:48.993080 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.993045    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" (UID: "c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:50:48.994559 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.994538    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-kube-api-access-r6xt6" (OuterVolumeSpecName: "kube-api-access-r6xt6") pod "c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" (UID: "c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa"). InnerVolumeSpecName "kube-api-access-r6xt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:50:48.994620 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:48.994568    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0cf30cd-6608-499b-9f91-a501080e5fae-kube-api-access-s2nf5" (OuterVolumeSpecName: "kube-api-access-s2nf5") pod "d0cf30cd-6608-499b-9f91-a501080e5fae" (UID: "d0cf30cd-6608-499b-9f91-a501080e5fae"). InnerVolumeSpecName "kube-api-access-s2nf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:50:49.098621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.098512    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:49.098621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.098557    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2nf5\" (UniqueName: \"kubernetes.io/projected/d0cf30cd-6608-499b-9f91-a501080e5fae-kube-api-access-s2nf5\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:49.098621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.098572    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0cf30cd-6608-499b-9f91-a501080e5fae-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:49.098621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.098584    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6xt6\" (UniqueName: \"kubernetes.io/projected/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa-kube-api-access-r6xt6\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:49.624724 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.624689    2563 generic.go:356] "Generic (PLEG): container finished" podID="c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" containerID="25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b" exitCode=0
Sep 08 15:50:49.625204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.624731    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"
Sep 08 15:50:49.625204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.624772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8" event={"ID":"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa","Type":"ContainerDied","Data":"25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b"}
Sep 08 15:50:49.625204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.624835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8" event={"ID":"c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa","Type":"ContainerDied","Data":"aabc84615a1dca5d2d91317bfd8c97110340d01ee02db723f6893cdf4e7db2cf"}
Sep 08 15:50:49.625204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.624866    2563 scope.go:117] "RemoveContainer" containerID="25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b"
Sep 08 15:50:49.626340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.626324    2563 generic.go:356] "Generic (PLEG): container finished" podID="d0cf30cd-6608-499b-9f91-a501080e5fae" containerID="2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8" exitCode=0
Sep 08 15:50:49.626436 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.626374    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"
Sep 08 15:50:49.626436 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.626405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz" event={"ID":"d0cf30cd-6608-499b-9f91-a501080e5fae","Type":"ContainerDied","Data":"2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8"}
Sep 08 15:50:49.626555 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.626466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz" event={"ID":"d0cf30cd-6608-499b-9f91-a501080e5fae","Type":"ContainerDied","Data":"cd2ab0aa0c13bdabdd1cc866abfc5653bd5757c3f415ea8ddb262a3c6f38695b"}
Sep 08 15:50:49.635558 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.635541    2563 scope.go:117] "RemoveContainer" containerID="25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b"
Sep 08 15:50:49.635862 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:50:49.635844    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b\": container with ID starting with 25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b not found: ID does not exist" containerID="25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b"
Sep 08 15:50:49.635920 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.635871    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b"} err="failed to get container status \"25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b\": rpc error: code = NotFound desc = could not find container \"25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b\": container with ID starting with 25f932f4e9033cbf9df64a43d7b1f8e09609e445d59c0225943aa095798cf60b not found: ID does not exist"
Sep 08 15:50:49.635920 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.635889    2563 scope.go:117] "RemoveContainer" containerID="2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8"
Sep 08 15:50:49.644353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.644337    2563 scope.go:117] "RemoveContainer" containerID="2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8"
Sep 08 15:50:49.644612 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:50:49.644591    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8\": container with ID starting with 2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8 not found: ID does not exist" containerID="2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8"
Sep 08 15:50:49.644708 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.644615    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8"} err="failed to get container status \"2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8\": rpc error: code = NotFound desc = could not find container \"2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8\": container with ID starting with 2bac56a1fd8d051b9c1014c16e7c65aaf5ab930dea4ec980e38f043a7aef17c8 not found: ID does not exist"
Sep 08 15:50:49.651170 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.651144    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"]
Sep 08 15:50:49.662345 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.662320    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-28e012-79c476cddb-8rfgz"]
Sep 08 15:50:49.677982 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.677956    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"]
Sep 08 15:50:49.684072 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:49.684049    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-28e012-6f6bdb4b9c-k24r8"]
Sep 08 15:50:50.102216 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:50.102184    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" path="/var/lib/kubelet/pods/c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa/volumes"
Sep 08 15:50:50.102652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:50.102637    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0cf30cd-6608-499b-9f91-a501080e5fae" path="/var/lib/kubelet/pods/d0cf30cd-6608-499b-9f91-a501080e5fae/volumes"
Sep 08 15:50:50.102938 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:50.102927    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dabf600d-5583-40e8-a2f4-f8565ae617f3" path="/var/lib/kubelet/pods/dabf600d-5583-40e8-a2f4-f8565ae617f3/volumes"
Sep 08 15:50:52.483841 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.483776    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k" podUID="62bf0655-f0c3-4307-861c-e4f95b33511d" containerName="main" containerID="cri-o://7c1ff33c6837c8b80ba032cf4df50cf384cb5a654909f1f7837c7829f13c975a" gracePeriod=45
Sep 08 15:50:52.653924 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.653879    2563 generic.go:356] "Generic (PLEG): container finished" podID="62bf0655-f0c3-4307-861c-e4f95b33511d" containerID="7c1ff33c6837c8b80ba032cf4df50cf384cb5a654909f1f7837c7829f13c975a" exitCode=0
Sep 08 15:50:52.654052 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.653958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k" event={"ID":"62bf0655-f0c3-4307-861c-e4f95b33511d","Type":"ContainerDied","Data":"7c1ff33c6837c8b80ba032cf4df50cf384cb5a654909f1f7837c7829f13c975a"}
Sep 08 15:50:52.740355 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.740270    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:50:52.835519 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xh7n\" (UniqueName: \"kubernetes.io/projected/62bf0655-f0c3-4307-861c-e4f95b33511d-kube-api-access-4xh7n\") pod \"62bf0655-f0c3-4307-861c-e4f95b33511d\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") "
Sep 08 15:50:52.835717 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835560    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-dshm\") pod \"62bf0655-f0c3-4307-861c-e4f95b33511d\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") "
Sep 08 15:50:52.835717 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835588    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-model-cache\") pod \"62bf0655-f0c3-4307-861c-e4f95b33511d\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") "
Sep 08 15:50:52.835816 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835775    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62bf0655-f0c3-4307-861c-e4f95b33511d-tls-certs\") pod \"62bf0655-f0c3-4307-861c-e4f95b33511d\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") "
Sep 08 15:50:52.835869 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835815    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-model-cache" (OuterVolumeSpecName: "model-cache") pod "62bf0655-f0c3-4307-861c-e4f95b33511d" (UID: "62bf0655-f0c3-4307-861c-e4f95b33511d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:50:52.835869 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835845    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-home\") pod \"62bf0655-f0c3-4307-861c-e4f95b33511d\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") "
Sep 08 15:50:52.835957 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.835930    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-tmp-dir\") pod \"62bf0655-f0c3-4307-861c-e4f95b33511d\" (UID: \"62bf0655-f0c3-4307-861c-e4f95b33511d\") "
Sep 08 15:50:52.836081 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.836055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-home" (OuterVolumeSpecName: "home") pod "62bf0655-f0c3-4307-861c-e4f95b33511d" (UID: "62bf0655-f0c3-4307-861c-e4f95b33511d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:50:52.836344 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.836323    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:52.836426 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.836351    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:52.836472 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.836450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "62bf0655-f0c3-4307-861c-e4f95b33511d" (UID: "62bf0655-f0c3-4307-861c-e4f95b33511d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:50:52.837937 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.837906    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62bf0655-f0c3-4307-861c-e4f95b33511d-kube-api-access-4xh7n" (OuterVolumeSpecName: "kube-api-access-4xh7n") pod "62bf0655-f0c3-4307-861c-e4f95b33511d" (UID: "62bf0655-f0c3-4307-861c-e4f95b33511d"). InnerVolumeSpecName "kube-api-access-4xh7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:50:52.837937 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.837924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62bf0655-f0c3-4307-861c-e4f95b33511d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "62bf0655-f0c3-4307-861c-e4f95b33511d" (UID: "62bf0655-f0c3-4307-861c-e4f95b33511d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:50:52.838074 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.837960    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-dshm" (OuterVolumeSpecName: "dshm") pod "62bf0655-f0c3-4307-861c-e4f95b33511d" (UID: "62bf0655-f0c3-4307-861c-e4f95b33511d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:50:52.937126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.937081    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:52.937126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.937121    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xh7n\" (UniqueName: \"kubernetes.io/projected/62bf0655-f0c3-4307-861c-e4f95b33511d-kube-api-access-4xh7n\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:52.937126 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.937133    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62bf0655-f0c3-4307-861c-e4f95b33511d-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:52.937423 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:52.937146    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62bf0655-f0c3-4307-861c-e4f95b33511d-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:53.660900 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:53.660868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k" event={"ID":"62bf0655-f0c3-4307-861c-e4f95b33511d","Type":"ContainerDied","Data":"33c720e56e08f1b875a8472974ffd9ac24be4c07145eb1a92ea4536e9c48b915"}
Sep 08 15:50:53.660900 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:53.660895    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"
Sep 08 15:50:53.661399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:53.660911    2563 scope.go:117] "RemoveContainer" containerID="7c1ff33c6837c8b80ba032cf4df50cf384cb5a654909f1f7837c7829f13c975a"
Sep 08 15:50:53.683449 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:53.683420    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"]
Sep 08 15:50:53.687337 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:53.687313    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a057c5/test-model-a057c5-kserve-596c98d8c-8p88k"]
Sep 08 15:50:54.100775 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:54.100741    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62bf0655-f0c3-4307-861c-e4f95b33511d" path="/var/lib/kubelet/pods/62bf0655-f0c3-4307-861c-e4f95b33511d/volumes"
Sep 08 15:50:55.506094 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.506055    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"]
Sep 08 15:50:55.506517 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.506277    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy" containerID="cri-o://c1dfcf90569826a1ff3aaf4a6857edfeafae0d42397a74d31aad25177956cda6" gracePeriod=30
Sep 08 15:50:55.871563 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871484    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"]
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871878    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dabf600d-5583-40e8-a2f4-f8565ae617f3" containerName="revoke-keys"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871888    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dabf600d-5583-40e8-a2f4-f8565ae617f3" containerName="revoke-keys"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871903    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0cf30cd-6608-499b-9f91-a501080e5fae" containerName="payload-pre-processing"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871909    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0cf30cd-6608-499b-9f91-a501080e5fae" containerName="payload-pre-processing"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871924    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62bf0655-f0c3-4307-861c-e4f95b33511d" containerName="main"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871929    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="62bf0655-f0c3-4307-861c-e4f95b33511d" containerName="main"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871938    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" containerName="payload-processing"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871944    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" containerName="payload-processing"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.871999    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dabf600d-5583-40e8-a2f4-f8565ae617f3" containerName="revoke-keys"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.872006    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0cf30cd-6608-499b-9f91-a501080e5fae" containerName="payload-pre-processing"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.872013    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="62bf0655-f0c3-4307-861c-e4f95b33511d" containerName="main"
Sep 08 15:50:55.872564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.872021    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b6c067-a4dd-462b-ad0c-d7e32ea4bfaa" containerName="payload-processing"
Sep 08 15:50:55.873705 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.873690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:50:55.875931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.875906    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:50:55.883463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.883438    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"]
Sep 08 15:50:55.964336 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:55.964268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx\" (UID: \"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:50:56.065156 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:56.065125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx\" (UID: \"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:50:56.065776 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:56.065755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx\" (UID: \"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:50:56.184548 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:56.184519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:50:56.456253 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:56.456220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"]
Sep 08 15:50:56.458766 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:50:56.458732    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6f3837_7866_4d6b_82f5_2c6ecef3c7c8.slice/crio-a03c0cd61acecc9236fdbc6f4846247f0f332bf14c88bbde17476e0ed1dd3b61 WatchSource:0}: Error finding container a03c0cd61acecc9236fdbc6f4846247f0f332bf14c88bbde17476e0ed1dd3b61: Status 404 returned error can't find the container with id a03c0cd61acecc9236fdbc6f4846247f0f332bf14c88bbde17476e0ed1dd3b61
Sep 08 15:50:56.679835 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:56.679802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx" event={"ID":"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8","Type":"ContainerStarted","Data":"a03c0cd61acecc9236fdbc6f4846247f0f332bf14c88bbde17476e0ed1dd3b61"}
Sep 08 15:50:57.686373 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:57.686341    2563 generic.go:356] "Generic (PLEG): container finished" podID="cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8" containerID="d2aca543b355226f0db71d0f27c1f5daea0990ec9b632b522b68788d4136e49f" exitCode=0
Sep 08 15:50:57.686750 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:57.686380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx" event={"ID":"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8","Type":"ContainerDied","Data":"d2aca543b355226f0db71d0f27c1f5daea0990ec9b632b522b68788d4136e49f"}
Sep 08 15:50:58.822618 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:58.822593    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:50:58.891992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:58.891958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8-maas-api-service-ca\") pod \"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8\" (UID: \"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8\") "
Sep 08 15:50:58.892413 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:58.892388    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8" (UID: "cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:50:58.993826 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:58.993730    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:50:59.698798 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:59.698758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx" event={"ID":"cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8","Type":"ContainerDied","Data":"a03c0cd61acecc9236fdbc6f4846247f0f332bf14c88bbde17476e0ed1dd3b61"}
Sep 08 15:50:59.698988 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:59.698814    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a03c0cd61acecc9236fdbc6f4846247f0f332bf14c88bbde17476e0ed1dd3b61"
Sep 08 15:50:59.698988 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:50:59.698768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"
Sep 08 15:51:00.706367 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.706337    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerID="c1dfcf90569826a1ff3aaf4a6857edfeafae0d42397a74d31aad25177956cda6" exitCode=0
Sep 08 15:51:00.706745 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.706397    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" event={"ID":"2d4a6f92-2453-4b66-9955-e14729c1c3fc","Type":"ContainerDied","Data":"c1dfcf90569826a1ff3aaf4a6857edfeafae0d42397a74d31aad25177956cda6"}
Sep 08 15:51:00.706745 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.706460    2563 scope.go:117] "RemoveContainer" containerID="d4e784b16af8c4346712a8bb186d8dcf916e887afab96646fdb508cb99cb7ef9"
Sep 08 15:51:00.744971 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.744947    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:51:00.813232 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcs48\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-kube-api-access-gcs48\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813232 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813197    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-envoy\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813486 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-certs\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813486 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813336    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-podinfo\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813486 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-socket\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813486 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istiod-ca-cert\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813700 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-data\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.813846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813545    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:51:00.814009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:51:00.814009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813808    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:51:00.814009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-token\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.814009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:51:00.814009 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.813990    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-credential-socket\") pod \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\" (UID: \"2d4a6f92-2453-4b66-9955-e14729c1c3fc\") "
Sep 08 15:51:00.814328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.814279    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:51:00.814514 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.814492    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.814578 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.814519    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.814578 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.814533    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.814578 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.814545    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.814578 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.814560    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.815625 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.815595    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:51:00.815728 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.815624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:51:00.815728 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.815631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-kube-api-access-gcs48" (OuterVolumeSpecName: "kube-api-access-gcs48") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "kube-api-access-gcs48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:51:00.815728 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.815632    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d4a6f92-2453-4b66-9955-e14729c1c3fc" (UID: "2d4a6f92-2453-4b66-9955-e14729c1c3fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:51:00.915315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.915252    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.915315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.915313    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcs48\" (UniqueName: \"kubernetes.io/projected/2d4a6f92-2453-4b66-9955-e14729c1c3fc-kube-api-access-gcs48\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.915315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.915324    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:00.915570 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:00.915333    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d4a6f92-2453-4b66-9955-e14729c1c3fc-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:01.713810 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:01.713777    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"
Sep 08 15:51:01.714326 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:01.713816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd" event={"ID":"2d4a6f92-2453-4b66-9955-e14729c1c3fc","Type":"ContainerDied","Data":"68ee13d60927623cba973081990357bf36d1493e8876081a47cfd270faa4f4cc"}
Sep 08 15:51:01.714326 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:01.713881    2563 scope.go:117] "RemoveContainer" containerID="c1dfcf90569826a1ff3aaf4a6857edfeafae0d42397a74d31aad25177956cda6"
Sep 08 15:51:01.734609 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:01.734582    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"]
Sep 08 15:51:01.738476 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:01.738450    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-28e012-openshift-default-7f79f998cd-vdpqd"]
Sep 08 15:51:02.106774 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:02.106672    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" path="/var/lib/kubelet/pods/2d4a6f92-2453-4b66-9955-e14729c1c3fc/volumes"
Sep 08 15:51:06.035063 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.035021    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"]
Sep 08 15:51:06.035489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.035306    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn" podUID="450ef3d2-9f02-404b-9377-d14ca9f5bf98" containerName="payload-processing" containerID="cri-o://5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b" gracePeriod=30
Sep 08 15:51:06.226740 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.226705    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"]
Sep 08 15:51:06.232967 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.232930    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a057c5-sjtzx"]
Sep 08 15:51:06.292473 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.292402    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:51:06.473981 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.473943    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7kr6\" (UniqueName: \"kubernetes.io/projected/450ef3d2-9f02-404b-9377-d14ca9f5bf98-kube-api-access-j7kr6\") pod \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") "
Sep 08 15:51:06.474176 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.474067    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450ef3d2-9f02-404b-9377-d14ca9f5bf98-plugins-config-volume\") pod \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\" (UID: \"450ef3d2-9f02-404b-9377-d14ca9f5bf98\") "
Sep 08 15:51:06.474566 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.474533    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/450ef3d2-9f02-404b-9377-d14ca9f5bf98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "450ef3d2-9f02-404b-9377-d14ca9f5bf98" (UID: "450ef3d2-9f02-404b-9377-d14ca9f5bf98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:51:06.476085 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.476061    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450ef3d2-9f02-404b-9377-d14ca9f5bf98-kube-api-access-j7kr6" (OuterVolumeSpecName: "kube-api-access-j7kr6") pod "450ef3d2-9f02-404b-9377-d14ca9f5bf98" (UID: "450ef3d2-9f02-404b-9377-d14ca9f5bf98"). InnerVolumeSpecName "kube-api-access-j7kr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:51:06.575483 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.575383    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450ef3d2-9f02-404b-9377-d14ca9f5bf98-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:06.575483 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.575419    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7kr6\" (UniqueName: \"kubernetes.io/projected/450ef3d2-9f02-404b-9377-d14ca9f5bf98-kube-api-access-j7kr6\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:51:06.741922 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.741889    2563 generic.go:356] "Generic (PLEG): container finished" podID="450ef3d2-9f02-404b-9377-d14ca9f5bf98" containerID="5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b" exitCode=0
Sep 08 15:51:06.742071 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.741938    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"
Sep 08 15:51:06.742071 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.741979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn" event={"ID":"450ef3d2-9f02-404b-9377-d14ca9f5bf98","Type":"ContainerDied","Data":"5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b"}
Sep 08 15:51:06.742071 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.742038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn" event={"ID":"450ef3d2-9f02-404b-9377-d14ca9f5bf98","Type":"ContainerDied","Data":"2bc15c5225fbf3c001eaf8eecb598bc400d523cba152db158c4fec13ee6be310"}
Sep 08 15:51:06.742071 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.742066    2563 scope.go:117] "RemoveContainer" containerID="5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b"
Sep 08 15:51:06.752023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.752002    2563 scope.go:117] "RemoveContainer" containerID="5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b"
Sep 08 15:51:06.752374 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:51:06.752351    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b\": container with ID starting with 5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b not found: ID does not exist" containerID="5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b"
Sep 08 15:51:06.752459 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.752386    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b"} err="failed to get container status \"5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b\": rpc error: code = NotFound desc = could not find container \"5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b\": container with ID starting with 5836f6eca78bd42514ca5a203ae9a99ced055a148f995bf8300ebab80c20771b not found: ID does not exist"
Sep 08 15:51:06.763560 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.763532    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"]
Sep 08 15:51:06.767113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:06.767090    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a057c5-5b9fff87bf-b2kdn"]
Sep 08 15:51:08.024136 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024099    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"]
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024509    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8" containerName="revoke-keys"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024522    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8" containerName="revoke-keys"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024537    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024542    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024559    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="450ef3d2-9f02-404b-9377-d14ca9f5bf98" containerName="payload-processing"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024565    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="450ef3d2-9f02-404b-9377-d14ca9f5bf98" containerName="payload-processing"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024625    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024633    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="450ef3d2-9f02-404b-9377-d14ca9f5bf98" containerName="payload-processing"
Sep 08 15:51:08.024664 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.024643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8" containerName="revoke-keys"
Sep 08 15:51:08.029087 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.029068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.031438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.031419    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-0ac386-openshift-default-dockercfg-n4qmp\""
Sep 08 15:51:08.042742 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.042715    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"]
Sep 08 15:51:08.100215 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.100180    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="450ef3d2-9f02-404b-9377-d14ca9f5bf98" path="/var/lib/kubelet/pods/450ef3d2-9f02-404b-9377-d14ca9f5bf98/volumes"
Sep 08 15:51:08.100588 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.100573    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8" path="/var/lib/kubelet/pods/cf6f3837-7866-4d6b-82f5-2c6ecef3c7c8/volumes"
Sep 08 15:51:08.190847 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.190808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wzzx\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-kube-api-access-6wzzx\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191061 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.190853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-data\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191061 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.190884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-token\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191061 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.190996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-socket\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191061 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.191036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073b8c8d-33e1-4436-89c9-ea7333d04512-istiod-ca-cert\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.191116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-envoy\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.191152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-podinfo\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191233 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.191189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-certs\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.191387 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.191232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-credential-socket\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-token\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-socket\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292264 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073b8c8d-33e1-4436-89c9-ea7333d04512-istiod-ca-cert\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-envoy\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-podinfo\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-certs\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-credential-socket\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wzzx\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-kube-api-access-6wzzx\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-data\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292913 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-socket\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292913 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292814    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-certs\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292913 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-data\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.292913 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.292893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-credential-socket\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.293088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.293066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073b8c8d-33e1-4436-89c9-ea7333d04512-istiod-ca-cert\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.295203 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.295174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-envoy\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.295533 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.295511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-podinfo\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.302036 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.302007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-token\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.306845 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.306821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wzzx\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-kube-api-access-6wzzx\") pod \"e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") " pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.339571 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.339539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:08.635918 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.635832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"]
Sep 08 15:51:08.640909 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:08.640878    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073b8c8d_33e1_4436_89c9_ea7333d04512.slice/crio-5e30fbb5e283401c628d6d816000c13b637f486e9a4d175dc843ca412a894d5f WatchSource:0}: Error finding container 5e30fbb5e283401c628d6d816000c13b637f486e9a4d175dc843ca412a894d5f: Status 404 returned error can't find the container with id 5e30fbb5e283401c628d6d816000c13b637f486e9a4d175dc843ca412a894d5f
Sep 08 15:51:08.643045 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.643013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:08.643097 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.643086    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:08.643130 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.643115    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:08.757229 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.757190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" event={"ID":"073b8c8d-33e1-4436-89c9-ea7333d04512","Type":"ContainerStarted","Data":"f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"}
Sep 08 15:51:08.757438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.757251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" event={"ID":"073b8c8d-33e1-4436-89c9-ea7333d04512","Type":"ContainerStarted","Data":"5e30fbb5e283401c628d6d816000c13b637f486e9a4d175dc843ca412a894d5f"}
Sep 08 15:51:08.779224 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:08.779163    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podStartSLOduration=1.7791458919999998 podStartE2EDuration="1.779145892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:08.77798234 +0000 UTC m=+1115.221174573" watchObservedRunningTime="2026-09-08 15:51:08.779145892 +0000 UTC m=+1115.222338100"
Sep 08 15:51:09.340354 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:09.340322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:09.345544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:09.345518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:09.763930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:09.763901    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:09.764804 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:09.764786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:12.564817 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.564780    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"]
Sep 08 15:51:12.565278 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.565201    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy"
Sep 08 15:51:12.565278 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.565212    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy"
Sep 08 15:51:12.565379 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.565300    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d4a6f92-2453-4b66-9955-e14729c1c3fc" containerName="istio-proxy"
Sep 08 15:51:12.568680 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.568662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.571072 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.571049    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-0ac386\""
Sep 08 15:51:12.579360 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.579331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"]
Sep 08 15:51:12.591644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.591619    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"]
Sep 08 15:51:12.595600 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.595581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.598037 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.598018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-0ac386-dockercfg-t9pmw\""
Sep 08 15:51:12.606062 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.606040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"]
Sep 08 15:51:12.637478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.637450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvgmn\" (UniqueName: \"kubernetes.io/projected/94067163-5806-4ce8-b835-9f37d4af8a57-kube-api-access-xvgmn\") pod \"payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.637636 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.637487    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88484dd2-781d-4d96-ba1a-093e42ede4c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.637636 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.637587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v922p\" (UniqueName: \"kubernetes.io/projected/88484dd2-781d-4d96-ba1a-093e42ede4c9-kube-api-access-v922p\") pod \"payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.637636 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.637635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94067163-5806-4ce8-b835-9f37d4af8a57-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.739140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.739084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v922p\" (UniqueName: \"kubernetes.io/projected/88484dd2-781d-4d96-ba1a-093e42ede4c9-kube-api-access-v922p\") pod \"payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.739140 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.739157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94067163-5806-4ce8-b835-9f37d4af8a57-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.739419 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.739203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvgmn\" (UniqueName: \"kubernetes.io/projected/94067163-5806-4ce8-b835-9f37d4af8a57-kube-api-access-xvgmn\") pod \"payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.739419 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.739228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88484dd2-781d-4d96-ba1a-093e42ede4c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.739906 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.739887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88484dd2-781d-4d96-ba1a-093e42ede4c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.739944 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.739913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94067163-5806-4ce8-b835-9f37d4af8a57-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.747087 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.747055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvgmn\" (UniqueName: \"kubernetes.io/projected/94067163-5806-4ce8-b835-9f37d4af8a57-kube-api-access-xvgmn\") pod \"payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.747211 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.747166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v922p\" (UniqueName: \"kubernetes.io/projected/88484dd2-781d-4d96-ba1a-093e42ede4c9-kube-api-access-v922p\") pod \"payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:12.881576 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.881483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:12.906441 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:12.906407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:13.158662 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.158638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"]
Sep 08 15:51:13.160597 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:13.160564    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94067163_5806_4ce8_b835_9f37d4af8a57.slice/crio-f5aafb1c473e592720e93ef30614e7ff68ccdcc129d82d422613ca5117fbd5a7 WatchSource:0}: Error finding container f5aafb1c473e592720e93ef30614e7ff68ccdcc129d82d422613ca5117fbd5a7: Status 404 returned error can't find the container with id f5aafb1c473e592720e93ef30614e7ff68ccdcc129d82d422613ca5117fbd5a7
Sep 08 15:51:13.189962 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.189871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"]
Sep 08 15:51:13.201423 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:13.201394    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88484dd2_781d_4d96_ba1a_093e42ede4c9.slice/crio-887f170deec0853f8497a09e3d1655f0a9796ae43adf1719b75d471607e57004 WatchSource:0}: Error finding container 887f170deec0853f8497a09e3d1655f0a9796ae43adf1719b75d471607e57004: Status 404 returned error can't find the container with id 887f170deec0853f8497a09e3d1655f0a9796ae43adf1719b75d471607e57004
Sep 08 15:51:13.790279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.790231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh" event={"ID":"94067163-5806-4ce8-b835-9f37d4af8a57","Type":"ContainerStarted","Data":"2bb57ab9308dd26f8493568658ee8c4510e837fc40b1aec8c568bc1a714d639a"}
Sep 08 15:51:13.790775 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.790321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:13.790775 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.790346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh" event={"ID":"94067163-5806-4ce8-b835-9f37d4af8a57","Type":"ContainerStarted","Data":"f5aafb1c473e592720e93ef30614e7ff68ccdcc129d82d422613ca5117fbd5a7"}
Sep 08 15:51:13.792478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.792418    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4" event={"ID":"88484dd2-781d-4d96-ba1a-093e42ede4c9","Type":"ContainerStarted","Data":"0b578996d836285d71e34cac6911558e5e66f345f27c91ae37962484816b14e3"}
Sep 08 15:51:13.792478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.792470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4" event={"ID":"88484dd2-781d-4d96-ba1a-093e42ede4c9","Type":"ContainerStarted","Data":"887f170deec0853f8497a09e3d1655f0a9796ae43adf1719b75d471607e57004"}
Sep 08 15:51:13.792647 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.792491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:13.810237 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.810179    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh" podStartSLOduration=1.8101618940000002 podStartE2EDuration="1.810161894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:13.808681005 +0000 UTC m=+1120.251873226" watchObservedRunningTime="2026-09-08 15:51:13.810161894 +0000 UTC m=+1120.253354103"
Sep 08 15:51:13.831669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:13.831617    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4" podStartSLOduration=1.831598971 podStartE2EDuration="1.831598971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:13.831226238 +0000 UTC m=+1120.274418496" watchObservedRunningTime="2026-09-08 15:51:13.831598971 +0000 UTC m=+1120.274791180"
Sep 08 15:51:14.621795 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.621761    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"]
Sep 08 15:51:14.626328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.626303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.628622 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.628590    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-b30e99-openshift-default-dockercfg-w6gz8\""
Sep 08 15:51:14.638580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.638553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"]
Sep 08 15:51:14.657787 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.657753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-certs\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.657946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.657810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-envoy\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.657946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.657864    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-data\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.657946 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.657898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thlq\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-kube-api-access-9thlq\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.658088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.657950    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-credential-socket\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.658088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.657995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-podinfo\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.658088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.658031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-token\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.658206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.658092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24daa250-2f29-470f-8f95-30a6a0d62e6e-istiod-ca-cert\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.658206 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.658165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-socket\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-socket\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-certs\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-envoy\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-data\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thlq\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-kube-api-access-9thlq\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759399 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-credential-socket\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-podinfo\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-token\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24daa250-2f29-470f-8f95-30a6a0d62e6e-istiod-ca-cert\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-socket\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.759669 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.759657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-certs\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.760033 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.760008    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-data\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.760181 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.760154    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-credential-socket\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.760272 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.760255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24daa250-2f29-470f-8f95-30a6a0d62e6e-istiod-ca-cert\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.762682 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.762651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-envoy\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.762837 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.762817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-podinfo\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.768311 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.768246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-token\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.768900 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.768877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thlq\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-kube-api-access-9thlq\") pod \"e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") " pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:14.940630 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:14.940588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:15.232557 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.232530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"]
Sep 08 15:51:15.234214 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:15.234178    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24daa250_2f29_470f_8f95_30a6a0d62e6e.slice/crio-59db92839365d158513856a39621dc0ffa827c0db72ea64bc7f7eaceb1afff47 WatchSource:0}: Error finding container 59db92839365d158513856a39621dc0ffa827c0db72ea64bc7f7eaceb1afff47: Status 404 returned error can't find the container with id 59db92839365d158513856a39621dc0ffa827c0db72ea64bc7f7eaceb1afff47
Sep 08 15:51:15.236631 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.236600    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:15.236694 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.236678    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:15.236737 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.236712    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:15.813104 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.813063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" event={"ID":"24daa250-2f29-470f-8f95-30a6a0d62e6e","Type":"ContainerStarted","Data":"9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"}
Sep 08 15:51:15.813273 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.813156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" event={"ID":"24daa250-2f29-470f-8f95-30a6a0d62e6e","Type":"ContainerStarted","Data":"59db92839365d158513856a39621dc0ffa827c0db72ea64bc7f7eaceb1afff47"}
Sep 08 15:51:15.835304 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.835240    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podStartSLOduration=1.8352218740000001 podStartE2EDuration="1.835221874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:15.834079558 +0000 UTC m=+1122.277271780" watchObservedRunningTime="2026-09-08 15:51:15.835221874 +0000 UTC m=+1122.278414090"
Sep 08 15:51:15.941046 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:15.941002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:16.946042 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:16.946014    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:17.826339 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:17.826282    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:17.827229 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:17.827209    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:19.687658 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.687612    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"]
Sep 08 15:51:19.691507 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.691485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.693930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.693909    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-b30e99\""
Sep 08 15:51:19.694052 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.694003    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-b30e99-dockercfg-7r6ls\""
Sep 08 15:51:19.702716 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.702690    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"]
Sep 08 15:51:19.774280 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.774237    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"]
Sep 08 15:51:19.777881 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.777851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:19.791344 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.791315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"]
Sep 08 15:51:19.813858 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.813818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps5k8\" (UniqueName: \"kubernetes.io/projected/54df6c43-c5af-4f4d-a001-366e586d9b00-kube-api-access-ps5k8\") pod \"payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:19.814065 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.813887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85245f1-c371-40c0-884b-372c86ede2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.814065 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.813949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54df6c43-c5af-4f4d-a001-366e586d9b00-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:19.814189 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.814108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncfrh\" (UniqueName: \"kubernetes.io/projected/d85245f1-c371-40c0-884b-372c86ede2b0-kube-api-access-ncfrh\") pod \"payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.915191 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.915152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5k8\" (UniqueName: \"kubernetes.io/projected/54df6c43-c5af-4f4d-a001-366e586d9b00-kube-api-access-ps5k8\") pod \"payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:19.915191 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.915198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85245f1-c371-40c0-884b-372c86ede2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.915512 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.915217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54df6c43-c5af-4f4d-a001-366e586d9b00-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:19.915512 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.915319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncfrh\" (UniqueName: \"kubernetes.io/projected/d85245f1-c371-40c0-884b-372c86ede2b0-kube-api-access-ncfrh\") pod \"payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.916044 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.916024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54df6c43-c5af-4f4d-a001-366e586d9b00-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:19.916096 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.916023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85245f1-c371-40c0-884b-372c86ede2b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.924210 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.924181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncfrh\" (UniqueName: \"kubernetes.io/projected/d85245f1-c371-40c0-884b-372c86ede2b0-kube-api-access-ncfrh\") pod \"payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:19.924408 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:19.924384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps5k8\" (UniqueName: \"kubernetes.io/projected/54df6c43-c5af-4f4d-a001-366e586d9b00-kube-api-access-ps5k8\") pod \"payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:20.003652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.003553    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:20.091665 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.091632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:20.316659 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.316629    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"]
Sep 08 15:51:20.318432 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:20.318401    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85245f1_c371_40c0_884b_372c86ede2b0.slice/crio-85087b4e07496f7efb453af6c4332a0ff670ddd0a19be481e84fbdfed2a12a6b WatchSource:0}: Error finding container 85087b4e07496f7efb453af6c4332a0ff670ddd0a19be481e84fbdfed2a12a6b: Status 404 returned error can't find the container with id 85087b4e07496f7efb453af6c4332a0ff670ddd0a19be481e84fbdfed2a12a6b
Sep 08 15:51:20.403759 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.403727    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"]
Sep 08 15:51:20.407131 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:20.407096    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54df6c43_c5af_4f4d_a001_366e586d9b00.slice/crio-a841321c9873c60c3ca7875d2aac84f03dd888681f777305bc8c510eed071efe WatchSource:0}: Error finding container a841321c9873c60c3ca7875d2aac84f03dd888681f777305bc8c510eed071efe: Status 404 returned error can't find the container with id a841321c9873c60c3ca7875d2aac84f03dd888681f777305bc8c510eed071efe
Sep 08 15:51:20.845371 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.845333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427" event={"ID":"54df6c43-c5af-4f4d-a001-366e586d9b00","Type":"ContainerStarted","Data":"45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267"}
Sep 08 15:51:20.846002 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.845967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427" event={"ID":"54df6c43-c5af-4f4d-a001-366e586d9b00","Type":"ContainerStarted","Data":"a841321c9873c60c3ca7875d2aac84f03dd888681f777305bc8c510eed071efe"}
Sep 08 15:51:20.847064 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.847036    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:20.849218 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.849191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm" event={"ID":"d85245f1-c371-40c0-884b-372c86ede2b0","Type":"ContainerStarted","Data":"1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9"}
Sep 08 15:51:20.849516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.849500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:20.849644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.849628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm" event={"ID":"d85245f1-c371-40c0-884b-372c86ede2b0","Type":"ContainerStarted","Data":"85087b4e07496f7efb453af6c4332a0ff670ddd0a19be481e84fbdfed2a12a6b"}
Sep 08 15:51:20.891962 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.891897    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm" podStartSLOduration=1.8918721029999999 podStartE2EDuration="1.891872103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:20.890022294 +0000 UTC m=+1127.333214528" watchObservedRunningTime="2026-09-08 15:51:20.891872103 +0000 UTC m=+1127.335064314"
Sep 08 15:51:20.892154 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:20.892014    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427" podStartSLOduration=1.8920054080000002 podStartE2EDuration="1.892005408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:20.868312255 +0000 UTC m=+1127.311504474" watchObservedRunningTime="2026-09-08 15:51:20.892005408 +0000 UTC m=+1127.335197620"
Sep 08 15:51:24.801241 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:24.801208    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:51:24.801661 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:24.801342    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:51:29.913353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:29.913315    2563 generic.go:356] "Generic (PLEG): container finished" podID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerID="f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90" exitCode=0
Sep 08 15:51:29.913914 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:29.913382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" event={"ID":"073b8c8d-33e1-4436-89c9-ea7333d04512","Type":"ContainerDied","Data":"f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"}
Sep 08 15:51:29.914003 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:29.913952    2563 scope.go:117] "RemoveContainer" containerID="f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"
Sep 08 15:51:29.915236 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:29.915204    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:29.915315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:29.915300    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:29.915358 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:29.915335    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:30.921658 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:30.921564    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" event={"ID":"073b8c8d-33e1-4436-89c9-ea7333d04512","Type":"ContainerStarted","Data":"ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0"}
Sep 08 15:51:31.340023 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:31.339979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:31.340353 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:31.340323    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 15:51:31.340409 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:31.340388    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 15:51:31.872539 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:31.872497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 15:51:32.340747 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:32.340708    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 15:51:32.341133 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:32.340766    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 15:51:32.869861 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:32.869825    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 15:51:33.340598 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:33.340559    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 15:51:33.340771 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:33.340616    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 15:51:33.868538 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:33.868481    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 15:51:33.868761 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:33.868569    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 15:51:34.340083 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.340050    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 15:51:34.341124 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.340115    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 15:51:34.969810 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.969770    2563 generic.go:356] "Generic (PLEG): container finished" podID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerID="9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f" exitCode=0
Sep 08 15:51:34.970039 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.969828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" event={"ID":"24daa250-2f29-470f-8f95-30a6a0d62e6e","Type":"ContainerDied","Data":"9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"}
Sep 08 15:51:34.970413 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.970391    2563 scope.go:117] "RemoveContainer" containerID="9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"
Sep 08 15:51:34.971624 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.971593    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:34.971736 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.971682    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:34.971736 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:34.971724    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 08 15:51:35.340411 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:35.340326    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 15:51:35.340411 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:35.340398    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 15:51:35.979559 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:35.979520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" event={"ID":"24daa250-2f29-470f-8f95-30a6a0d62e6e","Type":"ContainerStarted","Data":"deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e"}
Sep 08 15:51:36.355147 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:36.355039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:36.356451 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:36.356426    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:36.357393 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:36.357372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:51:36.940759 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:36.940723    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:36.940962 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:36.940922    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 08 15:51:36.941012 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:36.940968    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 08 15:51:37.941507 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:37.941467    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 08 15:51:37.941932 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:37.941530    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 08 15:51:38.941623 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:38.941587    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 08 15:51:38.942019 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:38.941646    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 08 15:51:39.941811 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:39.941773    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 08 15:51:39.942232 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:39.941843    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 08 15:51:40.941562 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:40.941520    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 08 15:51:40.941747 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:40.941585    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 08 15:51:41.944646 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:41.944617    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:41.945066 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:41.944876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:41.945516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:41.945496    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 15:51:48.447211 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.447174    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"]
Sep 08 15:51:48.451991 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.451971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.454403 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.454372    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-642d3b-dockercfg-qv6fb\""
Sep 08 15:51:48.454559 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.454535    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-642d3b\""
Sep 08 15:51:48.460730 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.460708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"]
Sep 08 15:51:48.470454 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.470427    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"]
Sep 08 15:51:48.482271 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.481789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.485593 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.485570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"]
Sep 08 15:51:48.539283 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.539251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.539452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.539303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5zhw\" (UniqueName: \"kubernetes.io/projected/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-kube-api-access-m5zhw\") pod \"payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.539452 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.539413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.539528 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.539452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwllt\" (UniqueName: \"kubernetes.io/projected/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-kube-api-access-xwllt\") pod \"payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.640990 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.640947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.641992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.641269    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5zhw\" (UniqueName: \"kubernetes.io/projected/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-kube-api-access-m5zhw\") pod \"payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.641992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.641430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.641992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.641546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwllt\" (UniqueName: \"kubernetes.io/projected/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-kube-api-access-xwllt\") pod \"payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.641992 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.641923    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.642695 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.642671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.650519 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.650490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwllt\" (UniqueName: \"kubernetes.io/projected/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-kube-api-access-xwllt\") pod \"payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:48.650666 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.650497    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5zhw\" (UniqueName: \"kubernetes.io/projected/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-kube-api-access-m5zhw\") pod \"payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.763735 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.763630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:48.793112 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:48.793065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:49.040770 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:49.040740    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"]
Sep 08 15:51:49.042179 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:49.042150    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b811684_42e7_4ca4_98ae_ccf0f31d73cf.slice/crio-d3d4771603440b8a1f1ccd37972b5365234b35fa866771aa6e974b43e7c13dcb WatchSource:0}: Error finding container d3d4771603440b8a1f1ccd37972b5365234b35fa866771aa6e974b43e7c13dcb: Status 404 returned error can't find the container with id d3d4771603440b8a1f1ccd37972b5365234b35fa866771aa6e974b43e7c13dcb
Sep 08 15:51:49.064939 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:49.064902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb" event={"ID":"0b811684-42e7-4ca4-98ae-ccf0f31d73cf","Type":"ContainerStarted","Data":"d3d4771603440b8a1f1ccd37972b5365234b35fa866771aa6e974b43e7c13dcb"}
Sep 08 15:51:49.077487 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:49.077454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"]
Sep 08 15:51:49.081590 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:51:49.081563    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c77a1f9_1f1b_4f52_8255_2f9d80ef7d76.slice/crio-efb397c9c2751d44dcf6427fbd7147f2c9638e95a2648efb15aad598c29722cf WatchSource:0}: Error finding container efb397c9c2751d44dcf6427fbd7147f2c9638e95a2648efb15aad598c29722cf: Status 404 returned error can't find the container with id efb397c9c2751d44dcf6427fbd7147f2c9638e95a2648efb15aad598c29722cf
Sep 08 15:51:50.073298 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.073247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd" event={"ID":"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76","Type":"ContainerStarted","Data":"485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690"}
Sep 08 15:51:50.073784 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.073336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd" event={"ID":"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76","Type":"ContainerStarted","Data":"efb397c9c2751d44dcf6427fbd7147f2c9638e95a2648efb15aad598c29722cf"}
Sep 08 15:51:50.073784 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.073369    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:51:50.074953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.074930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb" event={"ID":"0b811684-42e7-4ca4-98ae-ccf0f31d73cf","Type":"ContainerStarted","Data":"6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646"}
Sep 08 15:51:50.075113 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.074979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:51:50.089098 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.089052    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd" podStartSLOduration=2.089035009 podStartE2EDuration="2.089035009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:50.087562398 +0000 UTC m=+1156.530754624" watchObservedRunningTime="2026-09-08 15:51:50.089035009 +0000 UTC m=+1156.532227212"
Sep 08 15:51:50.102476 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:51:50.102430    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb" podStartSLOduration=2.102415857 podStartE2EDuration="2.102415857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:51:50.101234785 +0000 UTC m=+1156.544427009" watchObservedRunningTime="2026-09-08 15:51:50.102415857 +0000 UTC m=+1156.545608063"
Sep 08 15:52:01.083647 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:01.083564    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 15:52:01.083647 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:01.083634    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 15:52:34.224437 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:34.224401    2563 scope.go:117] "RemoveContainer" containerID="709afcee51cb0b3a68639861469d3b1f0f9d426d5b1d5683a0e075b97cb897bd"
Sep 08 15:52:59.518081 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.518038    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"]
Sep 08 15:52:59.523747 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.523725    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.526082 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.526057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-642d3b\"/\"default-dockercfg-sp62x\""
Sep 08 15:52:59.526210 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.526059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-642d3b\"/\"openshift-service-ca.crt\""
Sep 08 15:52:59.526849 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.526830    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-642d3b\"/\"kube-root-ca.crt\""
Sep 08 15:52:59.527077 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.527057    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-642d3b\"/\"auth-test-model-642d3b-kserve-self-signed-certs\""
Sep 08 15:52:59.531522 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.531498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"]
Sep 08 15:52:59.570979 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.570938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4zff\" (UniqueName: \"kubernetes.io/projected/0cd6723f-ddb7-4727-a345-70aa366e97b1-kube-api-access-f4zff\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.571138 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.570993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd6723f-ddb7-4727-a345-70aa366e97b1-tls-certs\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.571138 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.571032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-model-cache\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.571138 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.571055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-dshm\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.571138 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.571123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-home\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.571279 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.571206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-tmp-dir\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.671950 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.671909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-home\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672134 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.671994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-tmp-dir\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672134 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4zff\" (UniqueName: \"kubernetes.io/projected/0cd6723f-ddb7-4727-a345-70aa366e97b1-kube-api-access-f4zff\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672320 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd6723f-ddb7-4727-a345-70aa366e97b1-tls-certs\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672388 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-home\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672388 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-model-cache\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672482 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672387    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-tmp-dir\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672482 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-dshm\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.672696 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.672670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-model-cache\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.674615 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.674583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-dshm\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.674762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.674742    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd6723f-ddb7-4727-a345-70aa366e97b1-tls-certs\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.680562 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.680512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4zff\" (UniqueName: \"kubernetes.io/projected/0cd6723f-ddb7-4727-a345-70aa366e97b1-kube-api-access-f4zff\") pod \"auth-test-model-642d3b-kserve-7847dcc64c-mzhbm\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") " pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:52:59.836874 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:52:59.836782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:53:00.110988 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:00.110947    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"]
Sep 08 15:53:00.111896 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:53:00.111865    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cd6723f_ddb7_4727_a345_70aa366e97b1.slice/crio-397880b609eac272b012dddd4b97cb88f8962b8b904ed734a197d1d80210e2fa WatchSource:0}: Error finding container 397880b609eac272b012dddd4b97cb88f8962b8b904ed734a197d1d80210e2fa: Status 404 returned error can't find the container with id 397880b609eac272b012dddd4b97cb88f8962b8b904ed734a197d1d80210e2fa
Sep 08 15:53:00.486646 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:00.486604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm" event={"ID":"0cd6723f-ddb7-4727-a345-70aa366e97b1","Type":"ContainerStarted","Data":"dc7f321e36ddafbd1995a27ffb798cfacba98df9ff08c1fe52faf4f9cc045451"}
Sep 08 15:53:00.486836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:00.486659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm" event={"ID":"0cd6723f-ddb7-4727-a345-70aa366e97b1","Type":"ContainerStarted","Data":"397880b609eac272b012dddd4b97cb88f8962b8b904ed734a197d1d80210e2fa"}
Sep 08 15:53:00.524048 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:00.523994    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm" podStartSLOduration=1.52397921 podStartE2EDuration="1.52397921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:53:00.523447528 +0000 UTC m=+1226.966639736" watchObservedRunningTime="2026-09-08 15:53:00.52397921 +0000 UTC m=+1226.967171419"
Sep 08 15:53:02.711205 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.711169    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"]
Sep 08 15:53:02.714823 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.714806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:02.716844 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.716817    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:53:02.721131 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.721106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"]
Sep 08 15:53:02.796058 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.796009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c28d5e4-0f59-4643-bb14-04e6143398b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9\" (UID: \"2c28d5e4-0f59-4643-bb14-04e6143398b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:02.896736 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.896698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c28d5e4-0f59-4643-bb14-04e6143398b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9\" (UID: \"2c28d5e4-0f59-4643-bb14-04e6143398b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:02.897418 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:02.897392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c28d5e4-0f59-4643-bb14-04e6143398b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9\" (UID: \"2c28d5e4-0f59-4643-bb14-04e6143398b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:03.028619 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:03.028535    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:03.300837 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:03.300757    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"]
Sep 08 15:53:03.304255 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:53:03.304213    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c28d5e4_0f59_4643_bb14_04e6143398b9.slice/crio-589f588398802997d3ee562dbf8bb2ded1f528770b9ddf15bf66c1d48533edea WatchSource:0}: Error finding container 589f588398802997d3ee562dbf8bb2ded1f528770b9ddf15bf66c1d48533edea: Status 404 returned error can't find the container with id 589f588398802997d3ee562dbf8bb2ded1f528770b9ddf15bf66c1d48533edea
Sep 08 15:53:03.504851 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:03.504816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9" event={"ID":"2c28d5e4-0f59-4643-bb14-04e6143398b9","Type":"ContainerStarted","Data":"589f588398802997d3ee562dbf8bb2ded1f528770b9ddf15bf66c1d48533edea"}
Sep 08 15:53:04.512088 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:04.512049    2563 generic.go:356] "Generic (PLEG): container finished" podID="2c28d5e4-0f59-4643-bb14-04e6143398b9" containerID="c5600fbdf06ddfd2af38339301b2c9d1642e809bcfc67872b9940880efd89eec" exitCode=0
Sep 08 15:53:04.512600 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:04.512131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9" event={"ID":"2c28d5e4-0f59-4643-bb14-04e6143398b9","Type":"ContainerDied","Data":"c5600fbdf06ddfd2af38339301b2c9d1642e809bcfc67872b9940880efd89eec"}
Sep 08 15:53:05.656742 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:05.656710    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:05.722657 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:05.722621    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c28d5e4-0f59-4643-bb14-04e6143398b9-maas-api-service-ca\") pod \"2c28d5e4-0f59-4643-bb14-04e6143398b9\" (UID: \"2c28d5e4-0f59-4643-bb14-04e6143398b9\") "
Sep 08 15:53:05.722988 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:05.722965    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c28d5e4-0f59-4643-bb14-04e6143398b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c28d5e4-0f59-4643-bb14-04e6143398b9" (UID: "2c28d5e4-0f59-4643-bb14-04e6143398b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:53:05.823468 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:05.823373    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c28d5e4-0f59-4643-bb14-04e6143398b9-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:53:06.525947 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:06.525916    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"
Sep 08 15:53:06.526176 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:06.525959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9" event={"ID":"2c28d5e4-0f59-4643-bb14-04e6143398b9","Type":"ContainerDied","Data":"589f588398802997d3ee562dbf8bb2ded1f528770b9ddf15bf66c1d48533edea"}
Sep 08 15:53:06.526176 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:06.526006    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="589f588398802997d3ee562dbf8bb2ded1f528770b9ddf15bf66c1d48533edea"
Sep 08 15:53:09.837831 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:09.837786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:53:09.838268 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:09.837875    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:53:09.850899 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:09.850872    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:53:10.559540 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:10.559505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:53:18.364740 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:18.364695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"]
Sep 08 15:53:18.366796 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:18.366771    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-7eb8750f-jzgx9"]
Sep 08 15:53:20.101688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:20.101656    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c28d5e4-0f59-4643-bb14-04e6143398b9" path="/var/lib/kubelet/pods/2c28d5e4-0f59-4643-bb14-04e6143398b9/volumes"
Sep 08 15:53:34.250163 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:34.250091    2563 scope.go:117] "RemoveContainer" containerID="98a070e9bb7360e300dde3139b3ca4c20184e544340686663fc8d52e55e4091a"
Sep 08 15:53:34.258698 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:53:34.258678    2563 scope.go:117] "RemoveContainer" containerID="dd743781fc24b8d503e231a11a138c9fee63c8e27be2e181137610adc2d35125"
Sep 08 15:54:34.289957 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:54:34.289922    2563 scope.go:117] "RemoveContainer" containerID="9d8b1d0731f0d50ab7be45573dfd2052636b0a29439aee3b44ef44a08b0f516e"
Sep 08 15:54:34.299152 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:54:34.299129    2563 scope.go:117] "RemoveContainer" containerID="0829669cc0ad0c308f049ebac0f164a9e92dda3e714a82f86efd65449b6baf23"
Sep 08 15:55:02.109936 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:02.109844    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"]
Sep 08 15:55:17.131846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.131800    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm" podUID="0cd6723f-ddb7-4727-a345-70aa366e97b1" containerName="main" containerID="cri-o://dc7f321e36ddafbd1995a27ffb798cfacba98df9ff08c1fe52faf4f9cc045451" gracePeriod=45
Sep 08 15:55:17.275528 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.275488    2563 generic.go:356] "Generic (PLEG): container finished" podID="0cd6723f-ddb7-4727-a345-70aa366e97b1" containerID="dc7f321e36ddafbd1995a27ffb798cfacba98df9ff08c1fe52faf4f9cc045451" exitCode=0
Sep 08 15:55:17.275745 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.275545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm" event={"ID":"0cd6723f-ddb7-4727-a345-70aa366e97b1","Type":"ContainerDied","Data":"dc7f321e36ddafbd1995a27ffb798cfacba98df9ff08c1fe52faf4f9cc045451"}
Sep 08 15:55:17.380505 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.380476    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:55:17.481398 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-home\") pod \"0cd6723f-ddb7-4727-a345-70aa366e97b1\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") "
Sep 08 15:55:17.481611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481450    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-dshm\") pod \"0cd6723f-ddb7-4727-a345-70aa366e97b1\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") "
Sep 08 15:55:17.481611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-tmp-dir\") pod \"0cd6723f-ddb7-4727-a345-70aa366e97b1\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") "
Sep 08 15:55:17.481611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481547    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-model-cache\") pod \"0cd6723f-ddb7-4727-a345-70aa366e97b1\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") "
Sep 08 15:55:17.481898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd6723f-ddb7-4727-a345-70aa366e97b1-tls-certs\") pod \"0cd6723f-ddb7-4727-a345-70aa366e97b1\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") "
Sep 08 15:55:17.481898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481658    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-home" (OuterVolumeSpecName: "home") pod "0cd6723f-ddb7-4727-a345-70aa366e97b1" (UID: "0cd6723f-ddb7-4727-a345-70aa366e97b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:55:17.481898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481683    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4zff\" (UniqueName: \"kubernetes.io/projected/0cd6723f-ddb7-4727-a345-70aa366e97b1-kube-api-access-f4zff\") pod \"0cd6723f-ddb7-4727-a345-70aa366e97b1\" (UID: \"0cd6723f-ddb7-4727-a345-70aa366e97b1\") "
Sep 08 15:55:17.481898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0cd6723f-ddb7-4727-a345-70aa366e97b1" (UID: "0cd6723f-ddb7-4727-a345-70aa366e97b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:55:17.482171 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.481904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0cd6723f-ddb7-4727-a345-70aa366e97b1" (UID: "0cd6723f-ddb7-4727-a345-70aa366e97b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:55:17.482171 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.482122    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:55:17.482171 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.482141    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:55:17.482171 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.482155    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:55:17.484189 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.484152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cd6723f-ddb7-4727-a345-70aa366e97b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0cd6723f-ddb7-4727-a345-70aa366e97b1" (UID: "0cd6723f-ddb7-4727-a345-70aa366e97b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:55:17.484372 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.484348    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd6723f-ddb7-4727-a345-70aa366e97b1-kube-api-access-f4zff" (OuterVolumeSpecName: "kube-api-access-f4zff") pod "0cd6723f-ddb7-4727-a345-70aa366e97b1" (UID: "0cd6723f-ddb7-4727-a345-70aa366e97b1"). InnerVolumeSpecName "kube-api-access-f4zff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:55:17.484461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.484400    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-dshm" (OuterVolumeSpecName: "dshm") pod "0cd6723f-ddb7-4727-a345-70aa366e97b1" (UID: "0cd6723f-ddb7-4727-a345-70aa366e97b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:55:17.583217 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.583174    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cd6723f-ddb7-4727-a345-70aa366e97b1-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:55:17.583217 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.583211    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4zff\" (UniqueName: \"kubernetes.io/projected/0cd6723f-ddb7-4727-a345-70aa366e97b1-kube-api-access-f4zff\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:55:17.583217 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:17.583222    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cd6723f-ddb7-4727-a345-70aa366e97b1-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:55:18.282907 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:18.282880    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"
Sep 08 15:55:18.283432 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:18.282924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm" event={"ID":"0cd6723f-ddb7-4727-a345-70aa366e97b1","Type":"ContainerDied","Data":"397880b609eac272b012dddd4b97cb88f8962b8b904ed734a197d1d80210e2fa"}
Sep 08 15:55:18.283432 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:18.282976    2563 scope.go:117] "RemoveContainer" containerID="dc7f321e36ddafbd1995a27ffb798cfacba98df9ff08c1fe52faf4f9cc045451"
Sep 08 15:55:18.302149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:18.302117    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"]
Sep 08 15:55:18.306840 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:18.306811    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-642d3b/auth-test-model-642d3b-kserve-7847dcc64c-mzhbm"]
Sep 08 15:55:20.105246 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:20.105203    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cd6723f-ddb7-4727-a345-70aa366e97b1" path="/var/lib/kubelet/pods/0cd6723f-ddb7-4727-a345-70aa366e97b1/volumes"
Sep 08 15:55:34.325165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:34.325134    2563 scope.go:117] "RemoveContainer" containerID="c4a34aa86bcf067c94eacf190bf9dd8d083ad1d2ca181178e05ca654224b1ba5"
Sep 08 15:55:53.603856 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.603818    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"]
Sep 08 15:55:53.604306 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.604277    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd6723f-ddb7-4727-a345-70aa366e97b1" containerName="main"
Sep 08 15:55:53.604306 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.604304    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd6723f-ddb7-4727-a345-70aa366e97b1" containerName="main"
Sep 08 15:55:53.604394 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.604315    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c28d5e4-0f59-4643-bb14-04e6143398b9" containerName="revoke-keys"
Sep 08 15:55:53.604394 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.604320    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c28d5e4-0f59-4643-bb14-04e6143398b9" containerName="revoke-keys"
Sep 08 15:55:53.604461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.604407    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c28d5e4-0f59-4643-bb14-04e6143398b9" containerName="revoke-keys"
Sep 08 15:55:53.604461 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.604415    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd6723f-ddb7-4727-a345-70aa366e97b1" containerName="main"
Sep 08 15:55:53.606965 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.606949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.609749 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.609725    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9a16f5\""
Sep 08 15:55:53.609869 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.609758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9a16f5-dockercfg-gsmfz\""
Sep 08 15:55:53.615785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.615760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"]
Sep 08 15:55:53.654478 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.654443    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"]
Sep 08 15:55:53.656797 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.656780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.671595 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.671567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"]
Sep 08 15:55:53.746493 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.746461    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f835737d-1562-4ad3-9b15-c97d7763d54b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") " pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.746693 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.746501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkzm5\" (UniqueName: \"kubernetes.io/projected/f835737d-1562-4ad3-9b15-c97d7763d54b-kube-api-access-pkzm5\") pod \"payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") " pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.847484 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.847441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f835737d-1562-4ad3-9b15-c97d7763d54b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") " pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.847484 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.847491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkzm5\" (UniqueName: \"kubernetes.io/projected/f835737d-1562-4ad3-9b15-c97d7763d54b-kube-api-access-pkzm5\") pod \"payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") " pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.847737 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.847573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqqbj\" (UniqueName: \"kubernetes.io/projected/afaa14a5-d51a-4918-b089-d523f1adedd1-kube-api-access-tqqbj\") pod \"payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.847737 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.847626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa14a5-d51a-4918-b089-d523f1adedd1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.848227 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.848203    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f835737d-1562-4ad3-9b15-c97d7763d54b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") " pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.856931 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.856881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkzm5\" (UniqueName: \"kubernetes.io/projected/f835737d-1562-4ad3-9b15-c97d7763d54b-kube-api-access-pkzm5\") pod \"payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") " pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.918345 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.918311    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:53.948980 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.948948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa14a5-d51a-4918-b089-d523f1adedd1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.949133 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.949120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqbj\" (UniqueName: \"kubernetes.io/projected/afaa14a5-d51a-4918-b089-d523f1adedd1-kube-api-access-tqqbj\") pod \"payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.949636 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.949619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa14a5-d51a-4918-b089-d523f1adedd1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.957652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.957621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqqbj\" (UniqueName: \"kubernetes.io/projected/afaa14a5-d51a-4918-b089-d523f1adedd1-kube-api-access-tqqbj\") pod \"payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:53.966953 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:53.966692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:54.200842 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.200816    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"]
Sep 08 15:55:54.202493 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:55:54.202462    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf835737d_1562_4ad3_9b15_c97d7763d54b.slice/crio-cd174336c7edbcbcdd86f145c4b1c777dc461e8fefdb0581746f219bc661bed3 WatchSource:0}: Error finding container cd174336c7edbcbcdd86f145c4b1c777dc461e8fefdb0581746f219bc661bed3: Status 404 returned error can't find the container with id cd174336c7edbcbcdd86f145c4b1c777dc461e8fefdb0581746f219bc661bed3
Sep 08 15:55:54.253713 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.253687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"]
Sep 08 15:55:54.255019 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:55:54.254996    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafaa14a5_d51a_4918_b089_d523f1adedd1.slice/crio-8e983cb2e45bf9ee8a903d64eea699357b1d3ae5c5d5256056c80bcb2d5b32bd WatchSource:0}: Error finding container 8e983cb2e45bf9ee8a903d64eea699357b1d3ae5c5d5256056c80bcb2d5b32bd: Status 404 returned error can't find the container with id 8e983cb2e45bf9ee8a903d64eea699357b1d3ae5c5d5256056c80bcb2d5b32bd
Sep 08 15:55:54.498768 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.498673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7" event={"ID":"f835737d-1562-4ad3-9b15-c97d7763d54b","Type":"ContainerStarted","Data":"56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e"}
Sep 08 15:55:54.498768 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.498751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7" event={"ID":"f835737d-1562-4ad3-9b15-c97d7763d54b","Type":"ContainerStarted","Data":"cd174336c7edbcbcdd86f145c4b1c777dc461e8fefdb0581746f219bc661bed3"}
Sep 08 15:55:54.498998 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.498843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:55:54.500326 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.500275    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck" event={"ID":"afaa14a5-d51a-4918-b089-d523f1adedd1","Type":"ContainerStarted","Data":"e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2"}
Sep 08 15:55:54.500470 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.500354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:55:54.500470 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.500377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck" event={"ID":"afaa14a5-d51a-4918-b089-d523f1adedd1","Type":"ContainerStarted","Data":"8e983cb2e45bf9ee8a903d64eea699357b1d3ae5c5d5256056c80bcb2d5b32bd"}
Sep 08 15:55:54.518985 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.518932    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7" podStartSLOduration=1.518916608 podStartE2EDuration="1.518916608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:55:54.516945504 +0000 UTC m=+1400.960137983" watchObservedRunningTime="2026-09-08 15:55:54.518916608 +0000 UTC m=+1400.962108816"
Sep 08 15:55:54.533660 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:55:54.533609    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck" podStartSLOduration=1.533592098 podStartE2EDuration="1.533592098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:55:54.532463025 +0000 UTC m=+1400.975655261" watchObservedRunningTime="2026-09-08 15:55:54.533592098 +0000 UTC m=+1400.976784308"
Sep 08 15:56:05.508147 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:05.508112    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:56:05.508721 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:05.508196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:56:19.893375 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:19.893340    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"]
Sep 08 15:56:19.897099 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:19.897083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:19.899392 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:19.899362    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:56:19.904927 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:19.904899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"]
Sep 08 15:56:20.014797 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:20.014763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d3a4c07-7411-4555-b90f-f9427734784c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x\" (UID: \"9d3a4c07-7411-4555-b90f-f9427734784c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:20.116204 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:20.116169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d3a4c07-7411-4555-b90f-f9427734784c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x\" (UID: \"9d3a4c07-7411-4555-b90f-f9427734784c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:20.116877 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:20.116854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d3a4c07-7411-4555-b90f-f9427734784c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x\" (UID: \"9d3a4c07-7411-4555-b90f-f9427734784c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:20.210535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:20.210506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:20.483926 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:20.483897    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"]
Sep 08 15:56:20.486209 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:56:20.486180    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d3a4c07_7411_4555_b90f_f9427734784c.slice/crio-ed384874307b368db9fc70dec1a005da2c7025acef30cad52acf796eaa06fc01 WatchSource:0}: Error finding container ed384874307b368db9fc70dec1a005da2c7025acef30cad52acf796eaa06fc01: Status 404 returned error can't find the container with id ed384874307b368db9fc70dec1a005da2c7025acef30cad52acf796eaa06fc01
Sep 08 15:56:20.646225 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:20.646185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x" event={"ID":"9d3a4c07-7411-4555-b90f-f9427734784c","Type":"ContainerStarted","Data":"ed384874307b368db9fc70dec1a005da2c7025acef30cad52acf796eaa06fc01"}
Sep 08 15:56:21.653599 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:21.653561    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d3a4c07-7411-4555-b90f-f9427734784c" containerID="648398cfdb16297288ce8209ec3a1afff14c67554a58c5708ede3fa8b539d6fd" exitCode=0
Sep 08 15:56:21.654045 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:21.653645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x" event={"ID":"9d3a4c07-7411-4555-b90f-f9427734784c","Type":"ContainerDied","Data":"648398cfdb16297288ce8209ec3a1afff14c67554a58c5708ede3fa8b539d6fd"}
Sep 08 15:56:22.799863 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:22.799839    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:22.943365 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:22.943331    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d3a4c07-7411-4555-b90f-f9427734784c-maas-api-service-ca\") pod \"9d3a4c07-7411-4555-b90f-f9427734784c\" (UID: \"9d3a4c07-7411-4555-b90f-f9427734784c\") "
Sep 08 15:56:22.943695 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:22.943673    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d3a4c07-7411-4555-b90f-f9427734784c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d3a4c07-7411-4555-b90f-f9427734784c" (UID: "9d3a4c07-7411-4555-b90f-f9427734784c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:56:23.044671 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:23.044636    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d3a4c07-7411-4555-b90f-f9427734784c-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:56:23.666050 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:23.666024    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"
Sep 08 15:56:23.666050 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:23.666034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x" event={"ID":"9d3a4c07-7411-4555-b90f-f9427734784c","Type":"ContainerDied","Data":"ed384874307b368db9fc70dec1a005da2c7025acef30cad52acf796eaa06fc01"}
Sep 08 15:56:23.666270 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:23.666099    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed384874307b368db9fc70dec1a005da2c7025acef30cad52acf796eaa06fc01"
Sep 08 15:56:34.350238 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:34.350128    2563 scope.go:117] "RemoveContainer" containerID="5fb4d0991048acc52a0af611100b61d8a32193827492a1bec3a562935d55b5fb"
Sep 08 15:56:35.584344 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:35.584271    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"]
Sep 08 15:56:35.586892 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:35.586861    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dfa395-c6r7x"]
Sep 08 15:56:36.100923 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:36.100886    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d3a4c07-7411-4555-b90f-f9427734784c" path="/var/lib/kubelet/pods/9d3a4c07-7411-4555-b90f-f9427734784c/volumes"
Sep 08 15:56:42.742349 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.742308    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"]
Sep 08 15:56:42.742826 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.742810    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3a4c07-7411-4555-b90f-f9427734784c" containerName="revoke-keys"
Sep 08 15:56:42.742826 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.742828    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3a4c07-7411-4555-b90f-f9427734784c" containerName="revoke-keys"
Sep 08 15:56:42.742915 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.742906    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d3a4c07-7411-4555-b90f-f9427734784c" containerName="revoke-keys"
Sep 08 15:56:42.746331 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.746312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:42.749104 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.749080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:56:42.757544 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.757520    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"]
Sep 08 15:56:42.839854 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.839813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe182ee3-fd62-406a-80a1-0a381b1b9730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9a16f5-r246n\" (UID: \"fe182ee3-fd62-406a-80a1-0a381b1b9730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:42.941270 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.941229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe182ee3-fd62-406a-80a1-0a381b1b9730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9a16f5-r246n\" (UID: \"fe182ee3-fd62-406a-80a1-0a381b1b9730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:42.941878 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:42.941855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe182ee3-fd62-406a-80a1-0a381b1b9730-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9a16f5-r246n\" (UID: \"fe182ee3-fd62-406a-80a1-0a381b1b9730\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:43.057350 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:43.057260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:43.337990 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:43.337963    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"]
Sep 08 15:56:43.340044 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:56:43.340015    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe182ee3_fd62_406a_80a1_0a381b1b9730.slice/crio-4ebbf178d5d5860d0035f5b0d0ddf6983ce6fcad3f154ad6bd9f2d3ea0ebf383 WatchSource:0}: Error finding container 4ebbf178d5d5860d0035f5b0d0ddf6983ce6fcad3f154ad6bd9f2d3ea0ebf383: Status 404 returned error can't find the container with id 4ebbf178d5d5860d0035f5b0d0ddf6983ce6fcad3f154ad6bd9f2d3ea0ebf383
Sep 08 15:56:43.777384 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:43.777342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n" event={"ID":"fe182ee3-fd62-406a-80a1-0a381b1b9730","Type":"ContainerStarted","Data":"848e30f94c144333078bc597df0d3e8b8b1911696a50e83f1700f97ebafb97c9"}
Sep 08 15:56:43.777786 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:43.777411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n" event={"ID":"fe182ee3-fd62-406a-80a1-0a381b1b9730","Type":"ContainerStarted","Data":"4ebbf178d5d5860d0035f5b0d0ddf6983ce6fcad3f154ad6bd9f2d3ea0ebf383"}
Sep 08 15:56:43.805083 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:43.805027    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n" podStartSLOduration=1.805009771 podStartE2EDuration="1.805009771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:56:43.802585532 +0000 UTC m=+1450.245777758" watchObservedRunningTime="2026-09-08 15:56:43.805009771 +0000 UTC m=+1450.248201981"
Sep 08 15:56:44.784569 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:44.784537    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe182ee3-fd62-406a-80a1-0a381b1b9730" containerID="848e30f94c144333078bc597df0d3e8b8b1911696a50e83f1700f97ebafb97c9" exitCode=0
Sep 08 15:56:44.784975 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:44.784631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n" event={"ID":"fe182ee3-fd62-406a-80a1-0a381b1b9730","Type":"ContainerDied","Data":"848e30f94c144333078bc597df0d3e8b8b1911696a50e83f1700f97ebafb97c9"}
Sep 08 15:56:45.932394 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:45.932363    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:45.972281 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:45.972242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe182ee3-fd62-406a-80a1-0a381b1b9730-maas-api-service-ca\") pod \"fe182ee3-fd62-406a-80a1-0a381b1b9730\" (UID: \"fe182ee3-fd62-406a-80a1-0a381b1b9730\") "
Sep 08 15:56:45.972726 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:45.972695    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe182ee3-fd62-406a-80a1-0a381b1b9730-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe182ee3-fd62-406a-80a1-0a381b1b9730" (UID: "fe182ee3-fd62-406a-80a1-0a381b1b9730"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:56:46.073881 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:46.073795    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe182ee3-fd62-406a-80a1-0a381b1b9730-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:56:46.796312 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:46.796264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n" event={"ID":"fe182ee3-fd62-406a-80a1-0a381b1b9730","Type":"ContainerDied","Data":"4ebbf178d5d5860d0035f5b0d0ddf6983ce6fcad3f154ad6bd9f2d3ea0ebf383"}
Sep 08 15:56:46.796491 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:46.796332    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ebbf178d5d5860d0035f5b0d0ddf6983ce6fcad3f154ad6bd9f2d3ea0ebf383"
Sep 08 15:56:46.796491 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:46.796281    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"
Sep 08 15:56:58.216580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.216545    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"]
Sep 08 15:56:58.217115 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.216798    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7" podUID="f835737d-1562-4ad3-9b15-c97d7763d54b" containerName="payload-processing" containerID="cri-o://56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e" gracePeriod=30
Sep 08 15:56:58.223973 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.223835    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"]
Sep 08 15:56:58.224130 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.224100    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck" podUID="afaa14a5-d51a-4918-b089-d523f1adedd1" containerName="payload-pre-processing" containerID="cri-o://e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2" gracePeriod=30
Sep 08 15:56:58.388141 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.388085    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"]
Sep 08 15:56:58.390152 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.390130    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9a16f5-r246n"]
Sep 08 15:56:58.520940 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.520918    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:56:58.524165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.524146    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:56:58.605685 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.605659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f835737d-1562-4ad3-9b15-c97d7763d54b-plugins-config-volume\") pod \"f835737d-1562-4ad3-9b15-c97d7763d54b\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") "
Sep 08 15:56:58.605829 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.605724    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqqbj\" (UniqueName: \"kubernetes.io/projected/afaa14a5-d51a-4918-b089-d523f1adedd1-kube-api-access-tqqbj\") pod \"afaa14a5-d51a-4918-b089-d523f1adedd1\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") "
Sep 08 15:56:58.605873 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.605845    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkzm5\" (UniqueName: \"kubernetes.io/projected/f835737d-1562-4ad3-9b15-c97d7763d54b-kube-api-access-pkzm5\") pod \"f835737d-1562-4ad3-9b15-c97d7763d54b\" (UID: \"f835737d-1562-4ad3-9b15-c97d7763d54b\") "
Sep 08 15:56:58.605943 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.605925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa14a5-d51a-4918-b089-d523f1adedd1-plugins-config-volume\") pod \"afaa14a5-d51a-4918-b089-d523f1adedd1\" (UID: \"afaa14a5-d51a-4918-b089-d523f1adedd1\") "
Sep 08 15:56:58.606071 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.606050    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f835737d-1562-4ad3-9b15-c97d7763d54b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f835737d-1562-4ad3-9b15-c97d7763d54b" (UID: "f835737d-1562-4ad3-9b15-c97d7763d54b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:56:58.606315 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.606274    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afaa14a5-d51a-4918-b089-d523f1adedd1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "afaa14a5-d51a-4918-b089-d523f1adedd1" (UID: "afaa14a5-d51a-4918-b089-d523f1adedd1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:56:58.606441 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.606344    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f835737d-1562-4ad3-9b15-c97d7763d54b-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:56:58.607761 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.607740    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afaa14a5-d51a-4918-b089-d523f1adedd1-kube-api-access-tqqbj" (OuterVolumeSpecName: "kube-api-access-tqqbj") pod "afaa14a5-d51a-4918-b089-d523f1adedd1" (UID: "afaa14a5-d51a-4918-b089-d523f1adedd1"). InnerVolumeSpecName "kube-api-access-tqqbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:56:58.607982 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.607967    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f835737d-1562-4ad3-9b15-c97d7763d54b-kube-api-access-pkzm5" (OuterVolumeSpecName: "kube-api-access-pkzm5") pod "f835737d-1562-4ad3-9b15-c97d7763d54b" (UID: "f835737d-1562-4ad3-9b15-c97d7763d54b"). InnerVolumeSpecName "kube-api-access-pkzm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:56:58.707076 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.707032    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqqbj\" (UniqueName: \"kubernetes.io/projected/afaa14a5-d51a-4918-b089-d523f1adedd1-kube-api-access-tqqbj\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:56:58.707076 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.707067    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkzm5\" (UniqueName: \"kubernetes.io/projected/f835737d-1562-4ad3-9b15-c97d7763d54b-kube-api-access-pkzm5\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:56:58.707076 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.707078    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa14a5-d51a-4918-b089-d523f1adedd1-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:56:58.865190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.865096    2563 generic.go:356] "Generic (PLEG): container finished" podID="f835737d-1562-4ad3-9b15-c97d7763d54b" containerID="56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e" exitCode=0
Sep 08 15:56:58.865190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.865130    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"
Sep 08 15:56:58.865190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.865164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7" event={"ID":"f835737d-1562-4ad3-9b15-c97d7763d54b","Type":"ContainerDied","Data":"56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e"}
Sep 08 15:56:58.865505 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.865220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7" event={"ID":"f835737d-1562-4ad3-9b15-c97d7763d54b","Type":"ContainerDied","Data":"cd174336c7edbcbcdd86f145c4b1c777dc461e8fefdb0581746f219bc661bed3"}
Sep 08 15:56:58.865505 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.865250    2563 scope.go:117] "RemoveContainer" containerID="56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e"
Sep 08 15:56:58.866654 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.866630    2563 generic.go:356] "Generic (PLEG): container finished" podID="afaa14a5-d51a-4918-b089-d523f1adedd1" containerID="e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2" exitCode=0
Sep 08 15:56:58.866738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.866674    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"
Sep 08 15:56:58.866738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.866693    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck" event={"ID":"afaa14a5-d51a-4918-b089-d523f1adedd1","Type":"ContainerDied","Data":"e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2"}
Sep 08 15:56:58.866827 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.866756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck" event={"ID":"afaa14a5-d51a-4918-b089-d523f1adedd1","Type":"ContainerDied","Data":"8e983cb2e45bf9ee8a903d64eea699357b1d3ae5c5d5256056c80bcb2d5b32bd"}
Sep 08 15:56:58.879273 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.879249    2563 scope.go:117] "RemoveContainer" containerID="56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e"
Sep 08 15:56:58.879764 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:56:58.879740    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e\": container with ID starting with 56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e not found: ID does not exist" containerID="56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e"
Sep 08 15:56:58.879868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.879772    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e"} err="failed to get container status \"56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e\": rpc error: code = NotFound desc = could not find container \"56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e\": container with ID starting with 56bf5ed3241ea70b7da32c2c3b9e7cc5a874944fe397f7d0757dc1ac40223b4e not found: ID does not exist"
Sep 08 15:56:58.879868 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.879792    2563 scope.go:117] "RemoveContainer" containerID="e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2"
Sep 08 15:56:58.888873 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.888852    2563 scope.go:117] "RemoveContainer" containerID="e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2"
Sep 08 15:56:58.889141 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:56:58.889121    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2\": container with ID starting with e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2 not found: ID does not exist" containerID="e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2"
Sep 08 15:56:58.889187 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.889150    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2"} err="failed to get container status \"e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2\": rpc error: code = NotFound desc = could not find container \"e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2\": container with ID starting with e980c2286e48c97d343833d1e490d22d7cc25662af6addad41f5078595b9a5e2 not found: ID does not exist"
Sep 08 15:56:58.890387 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.890365    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"]
Sep 08 15:56:58.893763 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.893740    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9a16f5-74ccdf7f49-wlcz7"]
Sep 08 15:56:58.903598 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.903575    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"]
Sep 08 15:56:58.909803 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:56:58.909774    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9a16f5-665d989fcd-b5pck"]
Sep 08 15:57:00.101986 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:00.101950    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afaa14a5-d51a-4918-b089-d523f1adedd1" path="/var/lib/kubelet/pods/afaa14a5-d51a-4918-b089-d523f1adedd1/volumes"
Sep 08 15:57:00.102392 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:00.102351    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f835737d-1562-4ad3-9b15-c97d7763d54b" path="/var/lib/kubelet/pods/f835737d-1562-4ad3-9b15-c97d7763d54b/volumes"
Sep 08 15:57:00.102645 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:00.102633    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe182ee3-fd62-406a-80a1-0a381b1b9730" path="/var/lib/kubelet/pods/fe182ee3-fd62-406a-80a1-0a381b1b9730/volumes"
Sep 08 15:57:04.432379 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432344    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"]
Sep 08 15:57:04.432810 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432795    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f835737d-1562-4ad3-9b15-c97d7763d54b" containerName="payload-processing"
Sep 08 15:57:04.432810 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432806    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f835737d-1562-4ad3-9b15-c97d7763d54b" containerName="payload-processing"
Sep 08 15:57:04.432910 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432816    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afaa14a5-d51a-4918-b089-d523f1adedd1" containerName="payload-pre-processing"
Sep 08 15:57:04.432910 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432822    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="afaa14a5-d51a-4918-b089-d523f1adedd1" containerName="payload-pre-processing"
Sep 08 15:57:04.432910 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432829    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe182ee3-fd62-406a-80a1-0a381b1b9730" containerName="revoke-keys"
Sep 08 15:57:04.432910 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432834    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe182ee3-fd62-406a-80a1-0a381b1b9730" containerName="revoke-keys"
Sep 08 15:57:04.433073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432920    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe182ee3-fd62-406a-80a1-0a381b1b9730" containerName="revoke-keys"
Sep 08 15:57:04.433073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432926    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f835737d-1562-4ad3-9b15-c97d7763d54b" containerName="payload-processing"
Sep 08 15:57:04.433073 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.432937    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="afaa14a5-d51a-4918-b089-d523f1adedd1" containerName="payload-pre-processing"
Sep 08 15:57:04.435937 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.435917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.438686 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.438662    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b30e99\"/\"kube-root-ca.crt\""
Sep 08 15:57:04.439383 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.439361    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b30e99\"/\"default-dockercfg-hsn87\""
Sep 08 15:57:04.439498 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.439362    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b30e99\"/\"openshift-service-ca.crt\""
Sep 08 15:57:04.439793 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.439771    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b30e99\"/\"rate-test-model-b30e99-kserve-self-signed-certs\""
Sep 08 15:57:04.444518 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.444488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"]
Sep 08 15:57:04.568137 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.568092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-home\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.568340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.568153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/718bb433-ee49-45df-b05d-be0d9a50caf3-tls-certs\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.568340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.568200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-tmp-dir\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.568340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.568224    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-dshm\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.568340 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.568244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdd76\" (UniqueName: \"kubernetes.io/projected/718bb433-ee49-45df-b05d-be0d9a50caf3-kube-api-access-qdd76\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.568516 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.568454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-model-cache\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.669972 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.669933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/718bb433-ee49-45df-b05d-be0d9a50caf3-tls-certs\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.669972 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.669976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-tmp-dir\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670207 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-dshm\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670207 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdd76\" (UniqueName: \"kubernetes.io/projected/718bb433-ee49-45df-b05d-be0d9a50caf3-kube-api-access-qdd76\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670363 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-model-cache\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670363 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-home\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670484 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-tmp-dir\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-home\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.670756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.670736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-model-cache\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.672384 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.672351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-dshm\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.672634 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.672617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/718bb433-ee49-45df-b05d-be0d9a50caf3-tls-certs\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.677949 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.677923    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdd76\" (UniqueName: \"kubernetes.io/projected/718bb433-ee49-45df-b05d-be0d9a50caf3-kube-api-access-qdd76\") pod \"rate-test-model-b30e99-kserve-87d498c79-tsxnn\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") " pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:04.750830 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:04.750732    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:05.030629 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.030602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"]
Sep 08 15:57:05.032211 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:57:05.032176    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod718bb433_ee49_45df_b05d_be0d9a50caf3.slice/crio-a31805cb1055b3b089b63db5aee7e34c486c4952ce3cd0e932bb43f4f9032ffe WatchSource:0}: Error finding container a31805cb1055b3b089b63db5aee7e34c486c4952ce3cd0e932bb43f4f9032ffe: Status 404 returned error can't find the container with id a31805cb1055b3b089b63db5aee7e34c486c4952ce3cd0e932bb43f4f9032ffe
Sep 08 15:57:05.830935 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.830904    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"]
Sep 08 15:57:05.834744 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.834726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:05.836936 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.836915    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:57:05.842346 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.842321    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"]
Sep 08 15:57:05.908815 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.908779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn" event={"ID":"718bb433-ee49-45df-b05d-be0d9a50caf3","Type":"ContainerStarted","Data":"40213b0ae98d8a8816587c9274d3e8c28e55a3b9a2c7da47bfb56108c62f2445"}
Sep 08 15:57:05.908983 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.908828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn" event={"ID":"718bb433-ee49-45df-b05d-be0d9a50caf3","Type":"ContainerStarted","Data":"a31805cb1055b3b089b63db5aee7e34c486c4952ce3cd0e932bb43f4f9032ffe"}
Sep 08 15:57:05.926800 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.926750    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn" podStartSLOduration=1.926735299 podStartE2EDuration="1.926735299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:57:05.924336063 +0000 UTC m=+1472.367528276" watchObservedRunningTime="2026-09-08 15:57:05.926735299 +0000 UTC m=+1472.369927507"
Sep 08 15:57:05.983578 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:05.983537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/221283c9-ff7a-43b4-bb09-05c5542f3d9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p\" (UID: \"221283c9-ff7a-43b4-bb09-05c5542f3d9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:06.084639 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.084541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/221283c9-ff7a-43b4-bb09-05c5542f3d9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p\" (UID: \"221283c9-ff7a-43b4-bb09-05c5542f3d9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:06.085408 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.085388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/221283c9-ff7a-43b4-bb09-05c5542f3d9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p\" (UID: \"221283c9-ff7a-43b4-bb09-05c5542f3d9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:06.148174 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.148138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:06.419060 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.419036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"]
Sep 08 15:57:06.421047 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:57:06.421011    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221283c9_ff7a_43b4_bb09_05c5542f3d9f.slice/crio-91d34e7ba95c935092ddcdb9ff0e73d764e9c656a20249b1032849f20b09aebb WatchSource:0}: Error finding container 91d34e7ba95c935092ddcdb9ff0e73d764e9c656a20249b1032849f20b09aebb: Status 404 returned error can't find the container with id 91d34e7ba95c935092ddcdb9ff0e73d764e9c656a20249b1032849f20b09aebb
Sep 08 15:57:06.916109 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.916069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p" event={"ID":"221283c9-ff7a-43b4-bb09-05c5542f3d9f","Type":"ContainerStarted","Data":"54997e4579b57d44b19ca7c2dd6ecffef53da298c261d0d45fc43b6be010586c"}
Sep 08 15:57:06.916537 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.916129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p" event={"ID":"221283c9-ff7a-43b4-bb09-05c5542f3d9f","Type":"ContainerStarted","Data":"91d34e7ba95c935092ddcdb9ff0e73d764e9c656a20249b1032849f20b09aebb"}
Sep 08 15:57:06.931329 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:06.931254    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p" podStartSLOduration=1.931239724 podStartE2EDuration="1.931239724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:57:06.930403739 +0000 UTC m=+1473.373595961" watchObservedRunningTime="2026-09-08 15:57:06.931239724 +0000 UTC m=+1473.374431992"
Sep 08 15:57:07.923213 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:07.923176    2563 generic.go:356] "Generic (PLEG): container finished" podID="221283c9-ff7a-43b4-bb09-05c5542f3d9f" containerID="54997e4579b57d44b19ca7c2dd6ecffef53da298c261d0d45fc43b6be010586c" exitCode=0
Sep 08 15:57:07.923628 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:07.923219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p" event={"ID":"221283c9-ff7a-43b4-bb09-05c5542f3d9f","Type":"ContainerDied","Data":"54997e4579b57d44b19ca7c2dd6ecffef53da298c261d0d45fc43b6be010586c"}
Sep 08 15:57:09.068036 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.068010    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:09.114107 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.114063    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/221283c9-ff7a-43b4-bb09-05c5542f3d9f-maas-api-service-ca\") pod \"221283c9-ff7a-43b4-bb09-05c5542f3d9f\" (UID: \"221283c9-ff7a-43b4-bb09-05c5542f3d9f\") "
Sep 08 15:57:09.114430 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.114408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221283c9-ff7a-43b4-bb09-05c5542f3d9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "221283c9-ff7a-43b4-bb09-05c5542f3d9f" (UID: "221283c9-ff7a-43b4-bb09-05c5542f3d9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:09.215762 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.215666    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/221283c9-ff7a-43b4-bb09-05c5542f3d9f-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:09.937995 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.937961    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"
Sep 08 15:57:09.937995 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.937987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p" event={"ID":"221283c9-ff7a-43b4-bb09-05c5542f3d9f","Type":"ContainerDied","Data":"91d34e7ba95c935092ddcdb9ff0e73d764e9c656a20249b1032849f20b09aebb"}
Sep 08 15:57:09.938238 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:09.938060    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91d34e7ba95c935092ddcdb9ff0e73d764e9c656a20249b1032849f20b09aebb"
Sep 08 15:57:14.751392 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:14.751356    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:14.751849 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:14.751403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:14.764508 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:14.764480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:14.979846 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:14.979819    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:57:16.708602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:16.708559    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"]
Sep 08 15:57:16.711313 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:16.711280    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-be2aba-4mt6p"]
Sep 08 15:57:18.101149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:18.101117    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="221283c9-ff7a-43b4-bb09-05c5542f3d9f" path="/var/lib/kubelet/pods/221283c9-ff7a-43b4-bb09-05c5542f3d9f/volumes"
Sep 08 15:57:23.430470 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.430430    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"]
Sep 08 15:57:23.431103 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.431082    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="221283c9-ff7a-43b4-bb09-05c5542f3d9f" containerName="revoke-keys"
Sep 08 15:57:23.431194 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.431107    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="221283c9-ff7a-43b4-bb09-05c5542f3d9f" containerName="revoke-keys"
Sep 08 15:57:23.431245 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.431213    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="221283c9-ff7a-43b4-bb09-05c5542f3d9f" containerName="revoke-keys"
Sep 08 15:57:23.435714 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.435691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:23.438438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.438419    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 15:57:23.442334 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.442309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"]
Sep 08 15:57:23.563716 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.563681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f257db2-8447-45f2-804e-9a2f5fe60e02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7\" (UID: \"9f257db2-8447-45f2-804e-9a2f5fe60e02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:23.664352 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.664316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f257db2-8447-45f2-804e-9a2f5fe60e02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7\" (UID: \"9f257db2-8447-45f2-804e-9a2f5fe60e02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:23.664942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.664921    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f257db2-8447-45f2-804e-9a2f5fe60e02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7\" (UID: \"9f257db2-8447-45f2-804e-9a2f5fe60e02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:23.747898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:23.747806    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:24.032788 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:24.032708    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"]
Sep 08 15:57:24.036203 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:57:24.036172    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f257db2_8447_45f2_804e_9a2f5fe60e02.slice/crio-4fdc55a5ab6728b489c8a51ade11b50cec9a1bf385db4f4c670bf0cab61b09cf WatchSource:0}: Error finding container 4fdc55a5ab6728b489c8a51ade11b50cec9a1bf385db4f4c670bf0cab61b09cf: Status 404 returned error can't find the container with id 4fdc55a5ab6728b489c8a51ade11b50cec9a1bf385db4f4c670bf0cab61b09cf
Sep 08 15:57:25.032100 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:25.032008    2563 generic.go:356] "Generic (PLEG): container finished" podID="9f257db2-8447-45f2-804e-9a2f5fe60e02" containerID="88dfa0e0ea0077dd35a62f0285e71bfd13705163f4b12180d49295097d008713" exitCode=0
Sep 08 15:57:25.032500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:25.032098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7" event={"ID":"9f257db2-8447-45f2-804e-9a2f5fe60e02","Type":"ContainerDied","Data":"88dfa0e0ea0077dd35a62f0285e71bfd13705163f4b12180d49295097d008713"}
Sep 08 15:57:25.032500 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:25.032151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7" event={"ID":"9f257db2-8447-45f2-804e-9a2f5fe60e02","Type":"ContainerStarted","Data":"4fdc55a5ab6728b489c8a51ade11b50cec9a1bf385db4f4c670bf0cab61b09cf"}
Sep 08 15:57:26.174611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:26.174582    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:26.294323 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:26.294204    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f257db2-8447-45f2-804e-9a2f5fe60e02-maas-api-service-ca\") pod \"9f257db2-8447-45f2-804e-9a2f5fe60e02\" (UID: \"9f257db2-8447-45f2-804e-9a2f5fe60e02\") "
Sep 08 15:57:26.294615 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:26.294589    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f257db2-8447-45f2-804e-9a2f5fe60e02-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f257db2-8447-45f2-804e-9a2f5fe60e02" (UID: "9f257db2-8447-45f2-804e-9a2f5fe60e02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:26.395592 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:26.395548    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f257db2-8447-45f2-804e-9a2f5fe60e02-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:27.044621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:27.044584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7" event={"ID":"9f257db2-8447-45f2-804e-9a2f5fe60e02","Type":"ContainerDied","Data":"4fdc55a5ab6728b489c8a51ade11b50cec9a1bf385db4f4c670bf0cab61b09cf"}
Sep 08 15:57:27.044829 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:27.044642    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fdc55a5ab6728b489c8a51ade11b50cec9a1bf385db4f4c670bf0cab61b09cf"
Sep 08 15:57:27.044829 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:27.044591    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"
Sep 08 15:57:34.086937 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.086902    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"]
Sep 08 15:57:34.087756 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.087198    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4" podUID="88484dd2-781d-4d96-ba1a-093e42ede4c9" containerName="payload-processing" containerID="cri-o://0b578996d836285d71e34cac6911558e5e66f345f27c91ae37962484816b14e3" gracePeriod=30
Sep 08 15:57:34.091930 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.091900    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"]
Sep 08 15:57:34.092214 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.092168    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh" podUID="94067163-5806-4ce8-b835-9f37d4af8a57" containerName="payload-pre-processing" containerID="cri-o://2bb57ab9308dd26f8493568658ee8c4510e837fc40b1aec8c568bc1a714d639a" gracePeriod=30
Sep 08 15:57:34.238081 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.238041    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"]
Sep 08 15:57:34.241326 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.240974    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-0ac386-z7gr7"]
Sep 08 15:57:34.369661 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.369642    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:57:34.372190 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.372170    2563 scope.go:117] "RemoveContainer" containerID="d2aca543b355226f0db71d0f27c1f5daea0990ec9b632b522b68788d4136e49f"
Sep 08 15:57:34.373075 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.373054    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:57:34.380677 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.380660    2563 scope.go:117] "RemoveContainer" containerID="0b578996d836285d71e34cac6911558e5e66f345f27c91ae37962484816b14e3"
Sep 08 15:57:34.389911 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.389889    2563 scope.go:117] "RemoveContainer" containerID="2bb57ab9308dd26f8493568658ee8c4510e837fc40b1aec8c568bc1a714d639a"
Sep 08 15:57:34.400510 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.400491    2563 scope.go:117] "RemoveContainer" containerID="eb65b3166388ea862689288d0c7fd3bd8a5d05a1e571eeb8c3febb789f9b1a0c"
Sep 08 15:57:34.473534 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:57:34.473499    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94067163_5806_4ce8_b835_9f37d4af8a57.slice/crio-f5aafb1c473e592720e93ef30614e7ff68ccdcc129d82d422613ca5117fbd5a7\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88484dd2_781d_4d96_ba1a_093e42ede4c9.slice/crio-887f170deec0853f8497a09e3d1655f0a9796ae43adf1719b75d471607e57004\": RecentStats: unable to find data in memory cache]"
Sep 08 15:57:34.477898 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.477878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvgmn\" (UniqueName: \"kubernetes.io/projected/94067163-5806-4ce8-b835-9f37d4af8a57-kube-api-access-xvgmn\") pod \"94067163-5806-4ce8-b835-9f37d4af8a57\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") "
Sep 08 15:57:34.477991 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.477921    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94067163-5806-4ce8-b835-9f37d4af8a57-plugins-config-volume\") pod \"94067163-5806-4ce8-b835-9f37d4af8a57\" (UID: \"94067163-5806-4ce8-b835-9f37d4af8a57\") "
Sep 08 15:57:34.478038 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.478011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v922p\" (UniqueName: \"kubernetes.io/projected/88484dd2-781d-4d96-ba1a-093e42ede4c9-kube-api-access-v922p\") pod \"88484dd2-781d-4d96-ba1a-093e42ede4c9\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") "
Sep 08 15:57:34.478078 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.478044    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88484dd2-781d-4d96-ba1a-093e42ede4c9-plugins-config-volume\") pod \"88484dd2-781d-4d96-ba1a-093e42ede4c9\" (UID: \"88484dd2-781d-4d96-ba1a-093e42ede4c9\") "
Sep 08 15:57:34.478470 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.478441    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94067163-5806-4ce8-b835-9f37d4af8a57-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "94067163-5806-4ce8-b835-9f37d4af8a57" (UID: "94067163-5806-4ce8-b835-9f37d4af8a57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:34.478570 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.478463    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88484dd2-781d-4d96-ba1a-093e42ede4c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88484dd2-781d-4d96-ba1a-093e42ede4c9" (UID: "88484dd2-781d-4d96-ba1a-093e42ede4c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:34.480035 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.480011    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88484dd2-781d-4d96-ba1a-093e42ede4c9-kube-api-access-v922p" (OuterVolumeSpecName: "kube-api-access-v922p") pod "88484dd2-781d-4d96-ba1a-093e42ede4c9" (UID: "88484dd2-781d-4d96-ba1a-093e42ede4c9"). InnerVolumeSpecName "kube-api-access-v922p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:57:34.480035 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.480024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94067163-5806-4ce8-b835-9f37d4af8a57-kube-api-access-xvgmn" (OuterVolumeSpecName: "kube-api-access-xvgmn") pod "94067163-5806-4ce8-b835-9f37d4af8a57" (UID: "94067163-5806-4ce8-b835-9f37d4af8a57"). InnerVolumeSpecName "kube-api-access-xvgmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:57:34.579423 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.579394    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v922p\" (UniqueName: \"kubernetes.io/projected/88484dd2-781d-4d96-ba1a-093e42ede4c9-kube-api-access-v922p\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:34.579423 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.579417    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88484dd2-781d-4d96-ba1a-093e42ede4c9-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:34.579423 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.579427    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvgmn\" (UniqueName: \"kubernetes.io/projected/94067163-5806-4ce8-b835-9f37d4af8a57-kube-api-access-xvgmn\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:34.579629 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:34.579435    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94067163-5806-4ce8-b835-9f37d4af8a57-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:35.099847 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.099815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"
Sep 08 15:57:35.099847 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.099838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"
Sep 08 15:57:35.100328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.099813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh" event={"ID":"94067163-5806-4ce8-b835-9f37d4af8a57","Type":"ContainerDied","Data":"2bb57ab9308dd26f8493568658ee8c4510e837fc40b1aec8c568bc1a714d639a"}
Sep 08 15:57:35.100328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.099954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh" event={"ID":"94067163-5806-4ce8-b835-9f37d4af8a57","Type":"ContainerDied","Data":"f5aafb1c473e592720e93ef30614e7ff68ccdcc129d82d422613ca5117fbd5a7"}
Sep 08 15:57:35.100328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.099981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4" event={"ID":"88484dd2-781d-4d96-ba1a-093e42ede4c9","Type":"ContainerDied","Data":"0b578996d836285d71e34cac6911558e5e66f345f27c91ae37962484816b14e3"}
Sep 08 15:57:35.100328 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.100005    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4" event={"ID":"88484dd2-781d-4d96-ba1a-093e42ede4c9","Type":"ContainerDied","Data":"887f170deec0853f8497a09e3d1655f0a9796ae43adf1719b75d471607e57004"}
Sep 08 15:57:35.132668 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.132643    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"]
Sep 08 15:57:35.134737 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.134715    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-0ac386-59c4d6455d-bvwxh"]
Sep 08 15:57:35.158738 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.158717    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"]
Sep 08 15:57:35.162853 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:35.162835    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-0ac386-6859769877-4fkk4"]
Sep 08 15:57:36.102810 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:36.102766    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88484dd2-781d-4d96-ba1a-093e42ede4c9" path="/var/lib/kubelet/pods/88484dd2-781d-4d96-ba1a-093e42ede4c9/volumes"
Sep 08 15:57:36.103652 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:36.103626    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94067163-5806-4ce8-b835-9f37d4af8a57" path="/var/lib/kubelet/pods/94067163-5806-4ce8-b835-9f37d4af8a57/volumes"
Sep 08 15:57:36.104169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:36.104145    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f257db2-8447-45f2-804e-9a2f5fe60e02" path="/var/lib/kubelet/pods/9f257db2-8447-45f2-804e-9a2f5fe60e02/volumes"
Sep 08 15:57:40.545456 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:40.545422    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"]
Sep 08 15:57:40.545894 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:40.545643    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy" containerID="cri-o://ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0" gracePeriod=30
Sep 08 15:57:45.810409 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.810338    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:57:45.890529 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890498    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-envoy\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890542    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073b8c8d-33e1-4436-89c9-ea7333d04512-istiod-ca-cert\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890562    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-data\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890584    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-socket\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-credential-socket\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-token\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890683    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-certs\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.890732 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890712    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wzzx\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-kube-api-access-6wzzx\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.891092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890870    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-data" (OuterVolumeSpecName: "istio-data") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:45.891092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890946    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-podinfo\") pod \"073b8c8d-33e1-4436-89c9-ea7333d04512\" (UID: \"073b8c8d-33e1-4436-89c9-ea7333d04512\") "
Sep 08 15:57:45.891092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.890950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:45.891092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:45.891092 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/073b8c8d-33e1-4436-89c9-ea7333d04512-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 15:57:45.891332 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:45.891395 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891375    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073b8c8d-33e1-4436-89c9-ea7333d04512-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.891438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891400    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.891438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891413    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.891438 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891427    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.891580 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.891442    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.893297 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.893256    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:57:45.893428 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.893331    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-kube-api-access-6wzzx" (OuterVolumeSpecName: "kube-api-access-6wzzx") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "kube-api-access-6wzzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:57:45.893489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.893431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 15:57:45.893489 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.893443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-token" (OuterVolumeSpecName: "istio-token") pod "073b8c8d-33e1-4436-89c9-ea7333d04512" (UID: "073b8c8d-33e1-4436-89c9-ea7333d04512"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:57:45.992713 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.992676    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.992713 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.992709    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wzzx\" (UniqueName: \"kubernetes.io/projected/073b8c8d-33e1-4436-89c9-ea7333d04512-kube-api-access-6wzzx\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.992942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.992727    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:45.992942 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:45.992751    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073b8c8d-33e1-4436-89c9-ea7333d04512-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:57:46.169487 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.169447    2563 generic.go:356] "Generic (PLEG): container finished" podID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerID="ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0" exitCode=0
Sep 08 15:57:46.169681 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.169504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"
Sep 08 15:57:46.169681 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.169523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" event={"ID":"073b8c8d-33e1-4436-89c9-ea7333d04512","Type":"ContainerDied","Data":"ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0"}
Sep 08 15:57:46.169681 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.169575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d" event={"ID":"073b8c8d-33e1-4436-89c9-ea7333d04512","Type":"ContainerDied","Data":"5e30fbb5e283401c628d6d816000c13b637f486e9a4d175dc843ca412a894d5f"}
Sep 08 15:57:46.169681 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.169597    2563 scope.go:117] "RemoveContainer" containerID="ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0"
Sep 08 15:57:46.179053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.179035    2563 scope.go:117] "RemoveContainer" containerID="f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"
Sep 08 15:57:46.187502 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.187483    2563 scope.go:117] "RemoveContainer" containerID="ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0"
Sep 08 15:57:46.187801 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:57:46.187779    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0\": container with ID starting with ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0 not found: ID does not exist" containerID="ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0"
Sep 08 15:57:46.187853 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.187809    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0"} err="failed to get container status \"ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0\": rpc error: code = NotFound desc = could not find container \"ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0\": container with ID starting with ad88dfd6bc311dc2017dde075e8f98f639865ecc41f54f1aad0ed281a45f71d0 not found: ID does not exist"
Sep 08 15:57:46.187853 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.187828    2563 scope.go:117] "RemoveContainer" containerID="f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"
Sep 08 15:57:46.188109 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:57:46.188089    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90\": container with ID starting with f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90 not found: ID does not exist" containerID="f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"
Sep 08 15:57:46.188149 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.188116    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90"} err="failed to get container status \"f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90\": rpc error: code = NotFound desc = could not find container \"f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90\": container with ID starting with f2dd5d9e20ecae134aa1a2b90862a9ee9a56d945801e36a70f040ca5e4475c90 not found: ID does not exist"
Sep 08 15:57:46.191153 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.191125    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"]
Sep 08 15:57:46.195515 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:46.195494    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-0ac386-openshift-default-5fbf7cc998-j978d"]
Sep 08 15:57:48.101011 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:57:48.100973    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" path="/var/lib/kubelet/pods/073b8c8d-33e1-4436-89c9-ea7333d04512/volumes"
Sep 08 15:58:19.207249 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:19.207204    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"]
Sep 08 15:58:34.228539 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.228472    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn" podUID="718bb433-ee49-45df-b05d-be0d9a50caf3" containerName="main" containerID="cri-o://40213b0ae98d8a8816587c9274d3e8c28e55a3b9a2c7da47bfb56108c62f2445" gracePeriod=45
Sep 08 15:58:34.439247 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.439206    2563 generic.go:356] "Generic (PLEG): container finished" podID="718bb433-ee49-45df-b05d-be0d9a50caf3" containerID="40213b0ae98d8a8816587c9274d3e8c28e55a3b9a2c7da47bfb56108c62f2445" exitCode=0
Sep 08 15:58:34.439441 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.439270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn" event={"ID":"718bb433-ee49-45df-b05d-be0d9a50caf3","Type":"ContainerDied","Data":"40213b0ae98d8a8816587c9274d3e8c28e55a3b9a2c7da47bfb56108c62f2445"}
Sep 08 15:58:34.505468 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.505440    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:58:34.584044 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/718bb433-ee49-45df-b05d-be0d9a50caf3-tls-certs\") pod \"718bb433-ee49-45df-b05d-be0d9a50caf3\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") "
Sep 08 15:58:34.584261 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584101    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdd76\" (UniqueName: \"kubernetes.io/projected/718bb433-ee49-45df-b05d-be0d9a50caf3-kube-api-access-qdd76\") pod \"718bb433-ee49-45df-b05d-be0d9a50caf3\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") "
Sep 08 15:58:34.584261 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-dshm\") pod \"718bb433-ee49-45df-b05d-be0d9a50caf3\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") "
Sep 08 15:58:34.584261 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-tmp-dir\") pod \"718bb433-ee49-45df-b05d-be0d9a50caf3\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") "
Sep 08 15:58:34.584463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584273    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-model-cache\") pod \"718bb433-ee49-45df-b05d-be0d9a50caf3\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") "
Sep 08 15:58:34.584463 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584327    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-home\") pod \"718bb433-ee49-45df-b05d-be0d9a50caf3\" (UID: \"718bb433-ee49-45df-b05d-be0d9a50caf3\") "
Sep 08 15:58:34.584568 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584529    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "718bb433-ee49-45df-b05d-be0d9a50caf3" (UID: "718bb433-ee49-45df-b05d-be0d9a50caf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:34.584644 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584576    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-model-cache" (OuterVolumeSpecName: "model-cache") pod "718bb433-ee49-45df-b05d-be0d9a50caf3" (UID: "718bb433-ee49-45df-b05d-be0d9a50caf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:34.584691 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584655    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-home" (OuterVolumeSpecName: "home") pod "718bb433-ee49-45df-b05d-be0d9a50caf3" (UID: "718bb433-ee49-45df-b05d-be0d9a50caf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:34.584808 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584780    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-tmp-dir\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:58:34.584850 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584813    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-model-cache\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:58:34.584850 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.584826    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-home\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:58:34.586521 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.586496    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/718bb433-ee49-45df-b05d-be0d9a50caf3-kube-api-access-qdd76" (OuterVolumeSpecName: "kube-api-access-qdd76") pod "718bb433-ee49-45df-b05d-be0d9a50caf3" (UID: "718bb433-ee49-45df-b05d-be0d9a50caf3"). InnerVolumeSpecName "kube-api-access-qdd76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:58:34.586521 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.586506    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-dshm" (OuterVolumeSpecName: "dshm") pod "718bb433-ee49-45df-b05d-be0d9a50caf3" (UID: "718bb433-ee49-45df-b05d-be0d9a50caf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 15:58:34.586684 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.586517    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/718bb433-ee49-45df-b05d-be0d9a50caf3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "718bb433-ee49-45df-b05d-be0d9a50caf3" (UID: "718bb433-ee49-45df-b05d-be0d9a50caf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 15:58:34.686053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.686005    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/718bb433-ee49-45df-b05d-be0d9a50caf3-dshm\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:58:34.686053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.686047    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/718bb433-ee49-45df-b05d-be0d9a50caf3-tls-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:58:34.686053 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:34.686063    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdd76\" (UniqueName: \"kubernetes.io/projected/718bb433-ee49-45df-b05d-be0d9a50caf3-kube-api-access-qdd76\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:58:35.446195 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:35.446159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn" event={"ID":"718bb433-ee49-45df-b05d-be0d9a50caf3","Type":"ContainerDied","Data":"a31805cb1055b3b089b63db5aee7e34c486c4952ce3cd0e932bb43f4f9032ffe"}
Sep 08 15:58:35.446687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:35.446214    2563 scope.go:117] "RemoveContainer" containerID="40213b0ae98d8a8816587c9274d3e8c28e55a3b9a2c7da47bfb56108c62f2445"
Sep 08 15:58:35.446687 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:35.446176    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"
Sep 08 15:58:35.471797 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:35.471769    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"]
Sep 08 15:58:35.481935 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:35.481906    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b30e99/rate-test-model-b30e99-kserve-87d498c79-tsxnn"]
Sep 08 15:58:36.104482 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:36.102935    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="718bb433-ee49-45df-b05d-be0d9a50caf3" path="/var/lib/kubelet/pods/718bb433-ee49-45df-b05d-be0d9a50caf3/volumes"
Sep 08 15:58:55.060904 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.060866    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"]
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061306    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061319    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061331    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88484dd2-781d-4d96-ba1a-093e42ede4c9" containerName="payload-processing"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061337    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="88484dd2-781d-4d96-ba1a-093e42ede4c9" containerName="payload-processing"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061345    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f257db2-8447-45f2-804e-9a2f5fe60e02" containerName="revoke-keys"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061351    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f257db2-8447-45f2-804e-9a2f5fe60e02" containerName="revoke-keys"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061368    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94067163-5806-4ce8-b835-9f37d4af8a57" containerName="payload-pre-processing"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061374    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="94067163-5806-4ce8-b835-9f37d4af8a57" containerName="payload-pre-processing"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061381    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="718bb433-ee49-45df-b05d-be0d9a50caf3" containerName="main"
Sep 08 15:58:55.061397 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061386    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="718bb433-ee49-45df-b05d-be0d9a50caf3" containerName="main"
Sep 08 15:58:55.061725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061445    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy"
Sep 08 15:58:55.061725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061454    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="718bb433-ee49-45df-b05d-be0d9a50caf3" containerName="main"
Sep 08 15:58:55.061725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061462    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="88484dd2-781d-4d96-ba1a-093e42ede4c9" containerName="payload-processing"
Sep 08 15:58:55.061725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061469    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f257db2-8447-45f2-804e-9a2f5fe60e02" containerName="revoke-keys"
Sep 08 15:58:55.061725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061476    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="94067163-5806-4ce8-b835-9f37d4af8a57" containerName="payload-pre-processing"
Sep 08 15:58:55.061725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.061482    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy"
Sep 08 15:58:55.066122 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.066102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:58:55.093382 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.092511    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"]
Sep 08 15:58:55.194915 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.194868    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmnkq\" (UniqueName: \"kubernetes.io/projected/1aeecad6-53dc-458a-858c-8793e9eca93a-kube-api-access-gmnkq\") pod \"limitador-operator-controller-manager-74495c69f8-9r7kc\" (UID: \"1aeecad6-53dc-458a-858c-8793e9eca93a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:58:55.296129 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.296078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmnkq\" (UniqueName: \"kubernetes.io/projected/1aeecad6-53dc-458a-858c-8793e9eca93a-kube-api-access-gmnkq\") pod \"limitador-operator-controller-manager-74495c69f8-9r7kc\" (UID: \"1aeecad6-53dc-458a-858c-8793e9eca93a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:58:55.313938 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.313860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmnkq\" (UniqueName: \"kubernetes.io/projected/1aeecad6-53dc-458a-858c-8793e9eca93a-kube-api-access-gmnkq\") pod \"limitador-operator-controller-manager-74495c69f8-9r7kc\" (UID: \"1aeecad6-53dc-458a-858c-8793e9eca93a\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:58:55.377257 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.377218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:58:55.660169 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.660142    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"]
Sep 08 15:58:55.662323 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:58:55.662265    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aeecad6_53dc_458a_858c_8793e9eca93a.slice/crio-90caed64a3b50091f564e2f0018b9e4f67159264125b3f6c8f70f2f2d0fdaf23 WatchSource:0}: Error finding container 90caed64a3b50091f564e2f0018b9e4f67159264125b3f6c8f70f2f2d0fdaf23: Status 404 returned error can't find the container with id 90caed64a3b50091f564e2f0018b9e4f67159264125b3f6c8f70f2f2d0fdaf23
Sep 08 15:58:55.664667 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:55.664647    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 15:58:56.560173 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:56.560138    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc" event={"ID":"1aeecad6-53dc-458a-858c-8793e9eca93a","Type":"ContainerStarted","Data":"90caed64a3b50091f564e2f0018b9e4f67159264125b3f6c8f70f2f2d0fdaf23"}
Sep 08 15:58:57.568049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:57.567959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc" event={"ID":"1aeecad6-53dc-458a-858c-8793e9eca93a","Type":"ContainerStarted","Data":"5bf74df4c78170740568b25b98ff6462f7771201f2a328e23b538e14136afb22"}
Sep 08 15:58:57.568049 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:57.568024    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:58:57.591688 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:58:57.591635    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc" podStartSLOduration=0.993714485 podStartE2EDuration="2.591619303s" totalImagesPullingTime="1.597904818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:58:57.588695866 +0000 UTC m=+1584.031888075" watchObservedRunningTime="2026-09-08 15:58:57.591619303 +0000 UTC m=+1584.034811512"
Sep 08 15:59:07.585602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.585567    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"]
Sep 08 15:59:07.586036 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.586022    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy"
Sep 08 15:59:07.586036 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.586037    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="073b8c8d-33e1-4436-89c9-ea7333d04512" containerName="istio-proxy"
Sep 08 15:59:07.590895 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.589751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.594611 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.594588    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pmd7w\""
Sep 08 15:59:07.605430 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.605407    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"]
Sep 08 15:59:07.711705 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.711673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9s94\" (UniqueName: \"kubernetes.io/projected/b2c23c35-15ae-4612-a5b3-c357419f9cd8-kube-api-access-p9s94\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pg4dn\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.711867 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.711710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2c23c35-15ae-4612-a5b3-c357419f9cd8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pg4dn\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.813244 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.813207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9s94\" (UniqueName: \"kubernetes.io/projected/b2c23c35-15ae-4612-a5b3-c357419f9cd8-kube-api-access-p9s94\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pg4dn\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.813416 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.813248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2c23c35-15ae-4612-a5b3-c357419f9cd8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pg4dn\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.813618 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.813601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2c23c35-15ae-4612-a5b3-c357419f9cd8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pg4dn\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.821880 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.821856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9s94\" (UniqueName: \"kubernetes.io/projected/b2c23c35-15ae-4612-a5b3-c357419f9cd8-kube-api-access-p9s94\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pg4dn\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:07.904352 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:07.904263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:08.183621 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:08.183594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"]
Sep 08 15:59:08.185715 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:59:08.185679    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c23c35_15ae_4612_a5b3_c357419f9cd8.slice/crio-9ecc8282b6135b46e2126268fea5a8e5cd16a70c7f852d79ac8e7c23c6cb004c WatchSource:0}: Error finding container 9ecc8282b6135b46e2126268fea5a8e5cd16a70c7f852d79ac8e7c23c6cb004c: Status 404 returned error can't find the container with id 9ecc8282b6135b46e2126268fea5a8e5cd16a70c7f852d79ac8e7c23c6cb004c
Sep 08 15:59:08.576559 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:08.576468    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9r7kc"
Sep 08 15:59:08.631767 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:08.631728    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn" event={"ID":"b2c23c35-15ae-4612-a5b3-c357419f9cd8","Type":"ContainerStarted","Data":"9ecc8282b6135b46e2126268fea5a8e5cd16a70c7f852d79ac8e7c23c6cb004c"}
Sep 08 15:59:08.642039 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:08.642009    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"]
Sep 08 15:59:08.642228 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:08.642204    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn" podUID="2c95295b-1693-4305-ac64-5b55206b1b2b" containerName="manager" containerID="cri-o://3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f" gracePeriod=10
Sep 08 15:59:08.920258 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:08.920231    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:59:09.026785 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.026751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkd2c\" (UniqueName: \"kubernetes.io/projected/2c95295b-1693-4305-ac64-5b55206b1b2b-kube-api-access-wkd2c\") pod \"2c95295b-1693-4305-ac64-5b55206b1b2b\" (UID: \"2c95295b-1693-4305-ac64-5b55206b1b2b\") "
Sep 08 15:59:09.029165 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.029131    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c95295b-1693-4305-ac64-5b55206b1b2b-kube-api-access-wkd2c" (OuterVolumeSpecName: "kube-api-access-wkd2c") pod "2c95295b-1693-4305-ac64-5b55206b1b2b" (UID: "2c95295b-1693-4305-ac64-5b55206b1b2b"). InnerVolumeSpecName "kube-api-access-wkd2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 15:59:09.127892 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.127849    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkd2c\" (UniqueName: \"kubernetes.io/projected/2c95295b-1693-4305-ac64-5b55206b1b2b-kube-api-access-wkd2c\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 15:59:09.640116 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.640078    2563 generic.go:356] "Generic (PLEG): container finished" podID="2c95295b-1693-4305-ac64-5b55206b1b2b" containerID="3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f" exitCode=0
Sep 08 15:59:09.640116 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.640124    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"
Sep 08 15:59:09.640691 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.640162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn" event={"ID":"2c95295b-1693-4305-ac64-5b55206b1b2b","Type":"ContainerDied","Data":"3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f"}
Sep 08 15:59:09.640691 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.640231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn" event={"ID":"2c95295b-1693-4305-ac64-5b55206b1b2b","Type":"ContainerDied","Data":"1ba78cee444752623271647bef2da0eab6439b6b8ace9595ebd0a4ddb0f97814"}
Sep 08 15:59:09.640691 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.640261    2563 scope.go:117] "RemoveContainer" containerID="3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f"
Sep 08 15:59:09.651535 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.651511    2563 scope.go:117] "RemoveContainer" containerID="3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f"
Sep 08 15:59:09.651875 ip-10-0-135-240 kubenswrapper[2563]: E0908 15:59:09.651853    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f\": container with ID starting with 3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f not found: ID does not exist" containerID="3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f"
Sep 08 15:59:09.651972 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.651887    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f"} err="failed to get container status \"3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f\": rpc error: code = NotFound desc = could not find container \"3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f\": container with ID starting with 3816aed3d56a5d8afa81f1f2f74f055db6fef2a424a1c67db7f54a5a2602734f not found: ID does not exist"
Sep 08 15:59:09.668115 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.668084    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"]
Sep 08 15:59:09.670546 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.670519    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-wmfqn"]
Sep 08 15:59:09.722697 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.722640    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"]
Sep 08 15:59:09.723216 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.723197    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c95295b-1693-4305-ac64-5b55206b1b2b" containerName="manager"
Sep 08 15:59:09.723216 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.723218    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c95295b-1693-4305-ac64-5b55206b1b2b" containerName="manager"
Sep 08 15:59:09.723398 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.723366    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c95295b-1693-4305-ac64-5b55206b1b2b" containerName="manager"
Sep 08 15:59:09.727900 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.727874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:09.730333 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.730307    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 08 15:59:09.730458 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.730363    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-skph5\""
Sep 08 15:59:09.736887 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.736856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"]
Sep 08 15:59:09.834836 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.834800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk27q\" (UniqueName: \"kubernetes.io/projected/410e00e5-449b-43ad-8370-69fd2f853008-kube-api-access-kk27q\") pod \"dns-operator-controller-manager-7b84cd7c9c-wb4v8\" (UID: \"410e00e5-449b-43ad-8370-69fd2f853008\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:09.936132 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.936091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk27q\" (UniqueName: \"kubernetes.io/projected/410e00e5-449b-43ad-8370-69fd2f853008-kube-api-access-kk27q\") pod \"dns-operator-controller-manager-7b84cd7c9c-wb4v8\" (UID: \"410e00e5-449b-43ad-8370-69fd2f853008\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:09.947884 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:09.947853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk27q\" (UniqueName: \"kubernetes.io/projected/410e00e5-449b-43ad-8370-69fd2f853008-kube-api-access-kk27q\") pod \"dns-operator-controller-manager-7b84cd7c9c-wb4v8\" (UID: \"410e00e5-449b-43ad-8370-69fd2f853008\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:10.042322 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:10.042265    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:10.108309 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:10.107811    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c95295b-1693-4305-ac64-5b55206b1b2b" path="/var/lib/kubelet/pods/2c95295b-1693-4305-ac64-5b55206b1b2b/volumes"
Sep 08 15:59:10.328148 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:10.328114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"]
Sep 08 15:59:10.330672 ip-10-0-135-240 kubenswrapper[2563]: W0908 15:59:10.330641    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod410e00e5_449b_43ad_8370_69fd2f853008.slice/crio-aaeb01b9a563a3482043c29f1a0709bb552589011cea82ddb4c1a2412c754602 WatchSource:0}: Error finding container aaeb01b9a563a3482043c29f1a0709bb552589011cea82ddb4c1a2412c754602: Status 404 returned error can't find the container with id aaeb01b9a563a3482043c29f1a0709bb552589011cea82ddb4c1a2412c754602
Sep 08 15:59:10.648727 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:10.648633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8" event={"ID":"410e00e5-449b-43ad-8370-69fd2f853008","Type":"ContainerStarted","Data":"aaeb01b9a563a3482043c29f1a0709bb552589011cea82ddb4c1a2412c754602"}
Sep 08 15:59:12.663834 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:12.663799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn" event={"ID":"b2c23c35-15ae-4612-a5b3-c357419f9cd8","Type":"ContainerStarted","Data":"74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285"}
Sep 08 15:59:12.664319 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:12.663904    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:12.665564 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:12.665543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8" event={"ID":"410e00e5-449b-43ad-8370-69fd2f853008","Type":"ContainerStarted","Data":"f96ac60e57a0d50c539bb542ab786fa9e3d443115510cb8b8f7a64c8c0929fb5"}
Sep 08 15:59:12.665638 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:12.665624    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:12.693317 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:12.693254    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn" podStartSLOduration=2.164939537 podStartE2EDuration="5.693236109s" totalImagesPullingTime="3.528296572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:59:12.692006052 +0000 UTC m=+1599.135198301" watchObservedRunningTime="2026-09-08 15:59:12.693236109 +0000 UTC m=+1599.136428319"
Sep 08 15:59:12.721602 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:12.721550    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8" podStartSLOduration=1.505203753 podStartE2EDuration="3.721534055s" totalImagesPullingTime="2.216330302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 15:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 15:59:12.719550496 +0000 UTC m=+1599.162742705" watchObservedRunningTime="2026-09-08 15:59:12.721534055 +0000 UTC m=+1599.164726263"
Sep 08 15:59:23.231883 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:23.231855    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v54tt_3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878/payload-processing/0.log"
Sep 08 15:59:23.674177 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:23.674144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 15:59:23.674362 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:23.674221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wb4v8"
Sep 08 15:59:34.483818 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:34.483750    2563 scope.go:117] "RemoveContainer" containerID="c5600fbdf06ddfd2af38339301b2c9d1642e809bcfc67872b9940880efd89eec"
Sep 08 15:59:55.956725 ip-10-0-135-240 kubenswrapper[2563]: I0908 15:59:55.956694    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v54tt_3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878/payload-processing/0.log"
Sep 08 16:00:00.139129 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.139093    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"]
Sep 08 16:00:00.142225 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.142207    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"]
Sep 08 16:00:00.142387 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.142358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"
Sep 08 16:00:00.147331 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.145694    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 16:00:00.153812 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.153785    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"]
Sep 08 16:00:00.154894 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.154874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"
Sep 08 16:00:00.160505 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.158400    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"]
Sep 08 16:00:00.160505 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.158501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"
Sep 08 16:00:00.160505 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.160424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"]
Sep 08 16:00:00.163608 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.163545    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"]
Sep 08 16:00:00.169058 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.169038    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"
Sep 08 16:00:00.202267 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.202235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"
Sep 08 16:00:00.209656 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.209627    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"
Sep 08 16:00:00.453275 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.453242    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"]
Sep 08 16:00:00.455209 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:00:00.455178    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9b5cd0_cb67_45ba_a352_5c80f64a38c5.slice/crio-3e4fcfa75a06cbf313c1d4d9c7dddbb55006c82f597f82efe1630a5b14634e82 WatchSource:0}: Error finding container 3e4fcfa75a06cbf313c1d4d9c7dddbb55006c82f597f82efe1630a5b14634e82: Status 404 returned error can't find the container with id 3e4fcfa75a06cbf313c1d4d9c7dddbb55006c82f597f82efe1630a5b14634e82
Sep 08 16:00:00.497963 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.497938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"]
Sep 08 16:00:00.499499 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:00:00.499476    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7163289_ebb4_4905_831a_a0529822f69a.slice/crio-76d0881898a94cc31c591001515364ccfefd8c04dfb120b154ebf47272f03749 WatchSource:0}: Error finding container 76d0881898a94cc31c591001515364ccfefd8c04dfb120b154ebf47272f03749: Status 404 returned error can't find the container with id 76d0881898a94cc31c591001515364ccfefd8c04dfb120b154ebf47272f03749
Sep 08 16:00:00.500279 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:00:00.500262    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b71772a_7fbf_44c5_8d78_d52a850d1bd6.slice/crio-0355bbea730e99dc3dad5d2f5dc0b3f9feeac470aabeed94104bb8158604a0f9 WatchSource:0}: Error finding container 0355bbea730e99dc3dad5d2f5dc0b3f9feeac470aabeed94104bb8158604a0f9: Status 404 returned error can't find the container with id 0355bbea730e99dc3dad5d2f5dc0b3f9feeac470aabeed94104bb8158604a0f9
Sep 08 16:00:00.500486 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.500464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"]
Sep 08 16:00:00.948254 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.948216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv" event={"ID":"8b71772a-7fbf-44c5-8d78-d52a850d1bd6","Type":"ContainerStarted","Data":"b4515355bb5e22e9b92e7f710c85950f51dc48a205f952b299c2dac24cf2332d"}
Sep 08 16:00:00.948477 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.948273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv" event={"ID":"8b71772a-7fbf-44c5-8d78-d52a850d1bd6","Type":"ContainerStarted","Data":"0355bbea730e99dc3dad5d2f5dc0b3f9feeac470aabeed94104bb8158604a0f9"}
Sep 08 16:00:00.949897 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.949869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86" event={"ID":"8e9b5cd0-cb67-45ba-a352-5c80f64a38c5","Type":"ContainerStarted","Data":"417a3d9b8c521bf677f685c29b71dceeb91a0ebe4a63600bab222f491fd1cfdb"}
Sep 08 16:00:00.950039 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.949922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86" event={"ID":"8e9b5cd0-cb67-45ba-a352-5c80f64a38c5","Type":"ContainerStarted","Data":"3e4fcfa75a06cbf313c1d4d9c7dddbb55006c82f597f82efe1630a5b14634e82"}
Sep 08 16:00:00.951507 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.951485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl" event={"ID":"f7163289-ebb4-4905-831a-a0529822f69a","Type":"ContainerStarted","Data":"bdf5123677e9c8392f878ee507065c8c4f6dd93c6328a8d55a65a6f33c97add8"}
Sep 08 16:00:00.951601 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.951523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl" event={"ID":"f7163289-ebb4-4905-831a-a0529822f69a","Type":"ContainerStarted","Data":"76d0881898a94cc31c591001515364ccfefd8c04dfb120b154ebf47272f03749"}
Sep 08 16:00:00.976923 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:00.976875    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv" podStartSLOduration=0.976857626 podStartE2EDuration="976.857626ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:00:00.976182965 +0000 UTC m=+1647.419375177" watchObservedRunningTime="2026-09-08 16:00:00.976857626 +0000 UTC m=+1647.420049836"
Sep 08 16:00:01.000065 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.000013    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl" podStartSLOduration=0.99999756 podStartE2EDuration="999.99756ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:00:00.998078205 +0000 UTC m=+1647.441270413" watchObservedRunningTime="2026-09-08 16:00:00.99999756 +0000 UTC m=+1647.443189770"
Sep 08 16:00:01.024502 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.024382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86" podStartSLOduration=1.024366431 podStartE2EDuration="1.024366431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:00:01.023163091 +0000 UTC m=+1647.466355346" watchObservedRunningTime="2026-09-08 16:00:01.024366431 +0000 UTC m=+1647.467558640"
Sep 08 16:00:01.958755 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.958715    2563 generic.go:356] "Generic (PLEG): container finished" podID="f7163289-ebb4-4905-831a-a0529822f69a" containerID="bdf5123677e9c8392f878ee507065c8c4f6dd93c6328a8d55a65a6f33c97add8" exitCode=0
Sep 08 16:00:01.959216 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.958801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl" event={"ID":"f7163289-ebb4-4905-831a-a0529822f69a","Type":"ContainerDied","Data":"bdf5123677e9c8392f878ee507065c8c4f6dd93c6328a8d55a65a6f33c97add8"}
Sep 08 16:00:01.960485 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.960462    2563 generic.go:356] "Generic (PLEG): container finished" podID="8b71772a-7fbf-44c5-8d78-d52a850d1bd6" containerID="b4515355bb5e22e9b92e7f710c85950f51dc48a205f952b299c2dac24cf2332d" exitCode=0
Sep 08 16:00:01.960605 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.960531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv" event={"ID":"8b71772a-7fbf-44c5-8d78-d52a850d1bd6","Type":"ContainerDied","Data":"b4515355bb5e22e9b92e7f710c85950f51dc48a205f952b299c2dac24cf2332d"}
Sep 08 16:00:01.962054 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.962031    2563 generic.go:356] "Generic (PLEG): container finished" podID="8e9b5cd0-cb67-45ba-a352-5c80f64a38c5" containerID="417a3d9b8c521bf677f685c29b71dceeb91a0ebe4a63600bab222f491fd1cfdb" exitCode=0
Sep 08 16:00:01.962146 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:01.962086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86" event={"ID":"8e9b5cd0-cb67-45ba-a352-5c80f64a38c5","Type":"ContainerDied","Data":"417a3d9b8c521bf677f685c29b71dceeb91a0ebe4a63600bab222f491fd1cfdb"}
Sep 08 16:00:03.104558 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.104514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"
Sep 08 16:00:03.150496 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.150470    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"
Sep 08 16:00:03.185807 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.185779    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"
Sep 08 16:00:03.974999 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.974964    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"
Sep 08 16:00:03.974999 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.974976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv" event={"ID":"8b71772a-7fbf-44c5-8d78-d52a850d1bd6","Type":"ContainerDied","Data":"0355bbea730e99dc3dad5d2f5dc0b3f9feeac470aabeed94104bb8158604a0f9"}
Sep 08 16:00:03.975263 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.975053    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0355bbea730e99dc3dad5d2f5dc0b3f9feeac470aabeed94104bb8158604a0f9"
Sep 08 16:00:03.976277 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.976256    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"
Sep 08 16:00:03.976425 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.976317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86" event={"ID":"8e9b5cd0-cb67-45ba-a352-5c80f64a38c5","Type":"ContainerDied","Data":"3e4fcfa75a06cbf313c1d4d9c7dddbb55006c82f597f82efe1630a5b14634e82"}
Sep 08 16:00:03.976425 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.976359    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e4fcfa75a06cbf313c1d4d9c7dddbb55006c82f597f82efe1630a5b14634e82"
Sep 08 16:00:03.977851 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.977829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl" event={"ID":"f7163289-ebb4-4905-831a-a0529822f69a","Type":"ContainerDied","Data":"76d0881898a94cc31c591001515364ccfefd8c04dfb120b154ebf47272f03749"}
Sep 08 16:00:03.977959 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.977868    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76d0881898a94cc31c591001515364ccfefd8c04dfb120b154ebf47272f03749"
Sep 08 16:00:03.977959 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:03.977824    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814720-j8nsl"
Sep 08 16:00:26.141400 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141362    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"]
Sep 08 16:00:26.141845 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141831    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e9b5cd0-cb67-45ba-a352-5c80f64a38c5" containerName="cleanup"
Sep 08 16:00:26.141895 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141846    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e9b5cd0-cb67-45ba-a352-5c80f64a38c5" containerName="cleanup"
Sep 08 16:00:26.141895 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141859    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7163289-ebb4-4905-831a-a0529822f69a" containerName="cleanup"
Sep 08 16:00:26.141895 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141865    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7163289-ebb4-4905-831a-a0529822f69a" containerName="cleanup"
Sep 08 16:00:26.141895 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141872    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b71772a-7fbf-44c5-8d78-d52a850d1bd6" containerName="cleanup"
Sep 08 16:00:26.141895 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141878    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b71772a-7fbf-44c5-8d78-d52a850d1bd6" containerName="cleanup"
Sep 08 16:00:26.142056 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141942    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b71772a-7fbf-44c5-8d78-d52a850d1bd6" containerName="cleanup"
Sep 08 16:00:26.142056 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141949    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7163289-ebb4-4905-831a-a0529822f69a" containerName="cleanup"
Sep 08 16:00:26.142056 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.141957    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e9b5cd0-cb67-45ba-a352-5c80f64a38c5" containerName="cleanup"
Sep 08 16:00:26.143989 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.143971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:26.146327 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.146307    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 16:00:26.153022 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.152990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"]
Sep 08 16:00:26.219655 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.219615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bff8a9-9c07-40c9-bcad-3d3fe6223a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p\" (UID: \"66bff8a9-9c07-40c9-bcad-3d3fe6223a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:26.320190 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.320147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bff8a9-9c07-40c9-bcad-3d3fe6223a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p\" (UID: \"66bff8a9-9c07-40c9-bcad-3d3fe6223a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:26.320863 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.320840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bff8a9-9c07-40c9-bcad-3d3fe6223a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p\" (UID: \"66bff8a9-9c07-40c9-bcad-3d3fe6223a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:26.455893 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.455854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:26.725207 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:26.725175    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"]
Sep 08 16:00:26.726787 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:00:26.726760    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bff8a9_9c07_40c9_bcad_3d3fe6223a35.slice/crio-cbb0d93bd66b1990e6c356c2a12898f522193641239a1b7c145b441dc01d1eb4 WatchSource:0}: Error finding container cbb0d93bd66b1990e6c356c2a12898f522193641239a1b7c145b441dc01d1eb4: Status 404 returned error can't find the container with id cbb0d93bd66b1990e6c356c2a12898f522193641239a1b7c145b441dc01d1eb4
Sep 08 16:00:27.115036 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:27.114935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p" event={"ID":"66bff8a9-9c07-40c9-bcad-3d3fe6223a35","Type":"ContainerStarted","Data":"236ed4a4cf69882982f515397983e77216306327c7a44ec4feb3685536429dbe"}
Sep 08 16:00:27.115036 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:27.114995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p" event={"ID":"66bff8a9-9c07-40c9-bcad-3d3fe6223a35","Type":"ContainerStarted","Data":"cbb0d93bd66b1990e6c356c2a12898f522193641239a1b7c145b441dc01d1eb4"}
Sep 08 16:00:27.129801 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:27.129757    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p" podStartSLOduration=1.129735273 podStartE2EDuration="1.129735273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:00:27.129015102 +0000 UTC m=+1673.572207340" watchObservedRunningTime="2026-09-08 16:00:27.129735273 +0000 UTC m=+1673.572927519"
Sep 08 16:00:28.122277 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:28.122232    2563 generic.go:356] "Generic (PLEG): container finished" podID="66bff8a9-9c07-40c9-bcad-3d3fe6223a35" containerID="236ed4a4cf69882982f515397983e77216306327c7a44ec4feb3685536429dbe" exitCode=0
Sep 08 16:00:28.122672 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:28.122329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p" event={"ID":"66bff8a9-9c07-40c9-bcad-3d3fe6223a35","Type":"ContainerDied","Data":"236ed4a4cf69882982f515397983e77216306327c7a44ec4feb3685536429dbe"}
Sep 08 16:00:29.261970 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:29.261941    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:29.355553 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:29.355513    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bff8a9-9c07-40c9-bcad-3d3fe6223a35-maas-api-service-ca\") pod \"66bff8a9-9c07-40c9-bcad-3d3fe6223a35\" (UID: \"66bff8a9-9c07-40c9-bcad-3d3fe6223a35\") "
Sep 08 16:00:29.355856 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:29.355830    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66bff8a9-9c07-40c9-bcad-3d3fe6223a35-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66bff8a9-9c07-40c9-bcad-3d3fe6223a35" (UID: "66bff8a9-9c07-40c9-bcad-3d3fe6223a35"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:00:29.457052 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:29.457018    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66bff8a9-9c07-40c9-bcad-3d3fe6223a35-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:00:30.135226 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:30.135200    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"
Sep 08 16:00:30.135390 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:30.135186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p" event={"ID":"66bff8a9-9c07-40c9-bcad-3d3fe6223a35","Type":"ContainerDied","Data":"cbb0d93bd66b1990e6c356c2a12898f522193641239a1b7c145b441dc01d1eb4"}
Sep 08 16:00:30.135390 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:30.135339    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbb0d93bd66b1990e6c356c2a12898f522193641239a1b7c145b441dc01d1eb4"
Sep 08 16:00:32.508711 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:32.508649    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"]
Sep 08 16:00:32.511608 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:32.511579    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7a907b-jpg7p"]
Sep 08 16:00:34.101083 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:34.101052    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66bff8a9-9c07-40c9-bcad-3d3fe6223a35" path="/var/lib/kubelet/pods/66bff8a9-9c07-40c9-bcad-3d3fe6223a35/volumes"
Sep 08 16:00:42.551372 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.551333    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"]
Sep 08 16:00:42.551810 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.551796    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66bff8a9-9c07-40c9-bcad-3d3fe6223a35" containerName="revoke-keys"
Sep 08 16:00:42.551854 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.551814    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="66bff8a9-9c07-40c9-bcad-3d3fe6223a35" containerName="revoke-keys"
Sep 08 16:00:42.551916 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.551906    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="66bff8a9-9c07-40c9-bcad-3d3fe6223a35" containerName="revoke-keys"
Sep 08 16:00:42.554169 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.554147    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:42.556391 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.556370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 16:00:42.562521 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.562494    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"]
Sep 08 16:00:42.694802 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.694762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c6984dd-791a-4280-a814-34ffd4a764da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh\" (UID: \"1c6984dd-791a-4280-a814-34ffd4a764da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:42.796016 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.795976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c6984dd-791a-4280-a814-34ffd4a764da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh\" (UID: \"1c6984dd-791a-4280-a814-34ffd4a764da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:42.796646 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.796624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c6984dd-791a-4280-a814-34ffd4a764da-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh\" (UID: \"1c6984dd-791a-4280-a814-34ffd4a764da\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:42.865963 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:42.865861    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:43.139915 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:43.139886    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"]
Sep 08 16:00:43.142305 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:00:43.142260    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6984dd_791a_4280_a814_34ffd4a764da.slice/crio-d9d00881bb7ae58b3a5a5a32c68290e0b5ba56fbcf4df4d4ff1d18255186502a WatchSource:0}: Error finding container d9d00881bb7ae58b3a5a5a32c68290e0b5ba56fbcf4df4d4ff1d18255186502a: Status 404 returned error can't find the container with id d9d00881bb7ae58b3a5a5a32c68290e0b5ba56fbcf4df4d4ff1d18255186502a
Sep 08 16:00:43.210002 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:43.209964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh" event={"ID":"1c6984dd-791a-4280-a814-34ffd4a764da","Type":"ContainerStarted","Data":"d9d00881bb7ae58b3a5a5a32c68290e0b5ba56fbcf4df4d4ff1d18255186502a"}
Sep 08 16:00:44.217435 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:44.217395    2563 generic.go:356] "Generic (PLEG): container finished" podID="1c6984dd-791a-4280-a814-34ffd4a764da" containerID="0f6eb0300c7671af63c64a842ee12b19a8a6c42120c452a3e73f95a635d169ef" exitCode=0
Sep 08 16:00:44.217867 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:44.217440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh" event={"ID":"1c6984dd-791a-4280-a814-34ffd4a764da","Type":"ContainerDied","Data":"0f6eb0300c7671af63c64a842ee12b19a8a6c42120c452a3e73f95a635d169ef"}
Sep 08 16:00:45.369703 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:45.369673    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:45.423066 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:45.423025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c6984dd-791a-4280-a814-34ffd4a764da-maas-api-service-ca\") pod \"1c6984dd-791a-4280-a814-34ffd4a764da\" (UID: \"1c6984dd-791a-4280-a814-34ffd4a764da\") "
Sep 08 16:00:45.423466 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:45.423446    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c6984dd-791a-4280-a814-34ffd4a764da-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c6984dd-791a-4280-a814-34ffd4a764da" (UID: "1c6984dd-791a-4280-a814-34ffd4a764da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:00:45.524318 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:45.524209    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c6984dd-791a-4280-a814-34ffd4a764da-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:00:46.230943 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:46.230914    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"
Sep 08 16:00:46.231125 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:46.230917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh" event={"ID":"1c6984dd-791a-4280-a814-34ffd4a764da","Type":"ContainerDied","Data":"d9d00881bb7ae58b3a5a5a32c68290e0b5ba56fbcf4df4d4ff1d18255186502a"}
Sep 08 16:00:46.231125 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:46.231037    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9d00881bb7ae58b3a5a5a32c68290e0b5ba56fbcf4df4d4ff1d18255186502a"
Sep 08 16:00:58.041890 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:58.041794    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"]
Sep 08 16:00:58.046550 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:58.046521    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-e0da5a-29814720-nfb86"]
Sep 08 16:00:58.103058 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:58.103015    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e9b5cd0-cb67-45ba-a352-5c80f64a38c5" path="/var/lib/kubelet/pods/8e9b5cd0-cb67-45ba-a352-5c80f64a38c5/volumes"
Sep 08 16:00:58.222233 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:58.222190    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"]
Sep 08 16:00:58.225028 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:00:58.224996    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-e0da5a-dqfvh"]
Sep 08 16:01:00.102144 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:00.102113    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c6984dd-791a-4280-a814-34ffd4a764da" path="/var/lib/kubelet/pods/1c6984dd-791a-4280-a814-34ffd4a764da/volumes"
Sep 08 16:01:46.958226 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.958186    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"]
Sep 08 16:01:46.960763 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.958667    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6984dd-791a-4280-a814-34ffd4a764da" containerName="revoke-keys"
Sep 08 16:01:46.960763 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.958682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6984dd-791a-4280-a814-34ffd4a764da" containerName="revoke-keys"
Sep 08 16:01:46.960763 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.958762    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c6984dd-791a-4280-a814-34ffd4a764da" containerName="revoke-keys"
Sep 08 16:01:46.961829 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.961812    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:46.964045 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.964021    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcp9j\""
Sep 08 16:01:46.968784 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:46.968755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"]
Sep 08 16:01:47.053739 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:47.053704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd86630c-402b-49f4-9227-57ba3e08e1b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv\" (UID: \"dd86630c-402b-49f4-9227-57ba3e08e1b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:47.155123 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:47.155075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd86630c-402b-49f4-9227-57ba3e08e1b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv\" (UID: \"dd86630c-402b-49f4-9227-57ba3e08e1b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:47.155792 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:47.155763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd86630c-402b-49f4-9227-57ba3e08e1b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv\" (UID: \"dd86630c-402b-49f4-9227-57ba3e08e1b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:47.273186 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:47.273087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:47.578043 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:47.578016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"]
Sep 08 16:01:47.580780 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:01:47.580751    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd86630c_402b_49f4_9227_57ba3e08e1b7.slice/crio-4813242e9f3bb73160ad112160b06fd4dfad1e65e84723deb86ed369b5361001 WatchSource:0}: Error finding container 4813242e9f3bb73160ad112160b06fd4dfad1e65e84723deb86ed369b5361001: Status 404 returned error can't find the container with id 4813242e9f3bb73160ad112160b06fd4dfad1e65e84723deb86ed369b5361001
Sep 08 16:01:47.608416 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:47.608381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv" event={"ID":"dd86630c-402b-49f4-9227-57ba3e08e1b7","Type":"ContainerStarted","Data":"4813242e9f3bb73160ad112160b06fd4dfad1e65e84723deb86ed369b5361001"}
Sep 08 16:01:48.616128 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:48.616038    2563 generic.go:356] "Generic (PLEG): container finished" podID="dd86630c-402b-49f4-9227-57ba3e08e1b7" containerID="68ad5fd6b95ebf75b0941e8488d473caa3e5e969e0c9d6c32e3afc8271e80b82" exitCode=0
Sep 08 16:01:48.616541 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:48.616127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv" event={"ID":"dd86630c-402b-49f4-9227-57ba3e08e1b7","Type":"ContainerDied","Data":"68ad5fd6b95ebf75b0941e8488d473caa3e5e969e0c9d6c32e3afc8271e80b82"}
Sep 08 16:01:49.764626 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:49.764602    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:49.884116 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:49.883999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd86630c-402b-49f4-9227-57ba3e08e1b7-maas-api-service-ca\") pod \"dd86630c-402b-49f4-9227-57ba3e08e1b7\" (UID: \"dd86630c-402b-49f4-9227-57ba3e08e1b7\") "
Sep 08 16:01:49.884429 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:49.884404    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd86630c-402b-49f4-9227-57ba3e08e1b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dd86630c-402b-49f4-9227-57ba3e08e1b7" (UID: "dd86630c-402b-49f4-9227-57ba3e08e1b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:01:49.985279 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:49.985236    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd86630c-402b-49f4-9227-57ba3e08e1b7-maas-api-service-ca\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:01:50.628137 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:50.628102    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"
Sep 08 16:01:50.628353 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:50.628135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv" event={"ID":"dd86630c-402b-49f4-9227-57ba3e08e1b7","Type":"ContainerDied","Data":"4813242e9f3bb73160ad112160b06fd4dfad1e65e84723deb86ed369b5361001"}
Sep 08 16:01:50.628353 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:50.628191    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4813242e9f3bb73160ad112160b06fd4dfad1e65e84723deb86ed369b5361001"
Sep 08 16:01:57.660527 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:57.660486    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"]
Sep 08 16:01:57.662647 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:57.662626    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a3a348-ggjfv"]
Sep 08 16:01:58.101800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:01:58.101766    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd86630c-402b-49f4-9227-57ba3e08e1b7" path="/var/lib/kubelet/pods/dd86630c-402b-49f4-9227-57ba3e08e1b7/volumes"
Sep 08 16:02:15.198571 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.198520    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"]
Sep 08 16:02:15.202509 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.202470    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b30e99-29814720-kcqpv"]
Sep 08 16:02:15.234416 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.234369    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"]
Sep 08 16:02:15.234672 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.234623    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427" podUID="54df6c43-c5af-4f4d-a001-366e586d9b00" containerName="payload-processing" containerID="cri-o://45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267" gracePeriod=30
Sep 08 16:02:15.248016 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.247982    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"]
Sep 08 16:02:15.248263 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.248236    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm" podUID="d85245f1-c371-40c0-884b-372c86ede2b0" containerName="payload-pre-processing" containerID="cri-o://1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9" gracePeriod=30
Sep 08 16:02:15.540739 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.540708    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 16:02:15.543862 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.543839    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 16:02:15.638688 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.638652    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncfrh\" (UniqueName: \"kubernetes.io/projected/d85245f1-c371-40c0-884b-372c86ede2b0-kube-api-access-ncfrh\") pod \"d85245f1-c371-40c0-884b-372c86ede2b0\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") "
Sep 08 16:02:15.638864 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.638709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85245f1-c371-40c0-884b-372c86ede2b0-plugins-config-volume\") pod \"d85245f1-c371-40c0-884b-372c86ede2b0\" (UID: \"d85245f1-c371-40c0-884b-372c86ede2b0\") "
Sep 08 16:02:15.638864 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.638797    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54df6c43-c5af-4f4d-a001-366e586d9b00-plugins-config-volume\") pod \"54df6c43-c5af-4f4d-a001-366e586d9b00\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") "
Sep 08 16:02:15.638864 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.638860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps5k8\" (UniqueName: \"kubernetes.io/projected/54df6c43-c5af-4f4d-a001-366e586d9b00-kube-api-access-ps5k8\") pod \"54df6c43-c5af-4f4d-a001-366e586d9b00\" (UID: \"54df6c43-c5af-4f4d-a001-366e586d9b00\") "
Sep 08 16:02:15.639153 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.639122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d85245f1-c371-40c0-884b-372c86ede2b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d85245f1-c371-40c0-884b-372c86ede2b0" (UID: "d85245f1-c371-40c0-884b-372c86ede2b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:02:15.639346 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.639163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54df6c43-c5af-4f4d-a001-366e586d9b00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54df6c43-c5af-4f4d-a001-366e586d9b00" (UID: "54df6c43-c5af-4f4d-a001-366e586d9b00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:02:15.639346 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.639221    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85245f1-c371-40c0-884b-372c86ede2b0-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:15.640869 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.640842    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85245f1-c371-40c0-884b-372c86ede2b0-kube-api-access-ncfrh" (OuterVolumeSpecName: "kube-api-access-ncfrh") pod "d85245f1-c371-40c0-884b-372c86ede2b0" (UID: "d85245f1-c371-40c0-884b-372c86ede2b0"). InnerVolumeSpecName "kube-api-access-ncfrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:15.641054 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.641028    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54df6c43-c5af-4f4d-a001-366e586d9b00-kube-api-access-ps5k8" (OuterVolumeSpecName: "kube-api-access-ps5k8") pod "54df6c43-c5af-4f4d-a001-366e586d9b00" (UID: "54df6c43-c5af-4f4d-a001-366e586d9b00"). InnerVolumeSpecName "kube-api-access-ps5k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:15.740685 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.740584    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54df6c43-c5af-4f4d-a001-366e586d9b00-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:15.740685 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.740642    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps5k8\" (UniqueName: \"kubernetes.io/projected/54df6c43-c5af-4f4d-a001-366e586d9b00-kube-api-access-ps5k8\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:15.740685 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.740652    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncfrh\" (UniqueName: \"kubernetes.io/projected/d85245f1-c371-40c0-884b-372c86ede2b0-kube-api-access-ncfrh\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:15.775231 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.775189    2563 generic.go:356] "Generic (PLEG): container finished" podID="54df6c43-c5af-4f4d-a001-366e586d9b00" containerID="45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267" exitCode=0
Sep 08 16:02:15.775454 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.775237    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"
Sep 08 16:02:15.775454 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.775272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427" event={"ID":"54df6c43-c5af-4f4d-a001-366e586d9b00","Type":"ContainerDied","Data":"45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267"}
Sep 08 16:02:15.775454 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.775396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427" event={"ID":"54df6c43-c5af-4f4d-a001-366e586d9b00","Type":"ContainerDied","Data":"a841321c9873c60c3ca7875d2aac84f03dd888681f777305bc8c510eed071efe"}
Sep 08 16:02:15.775454 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.775434    2563 scope.go:117] "RemoveContainer" containerID="45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267"
Sep 08 16:02:15.776905 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.776882    2563 generic.go:356] "Generic (PLEG): container finished" podID="d85245f1-c371-40c0-884b-372c86ede2b0" containerID="1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9" exitCode=0
Sep 08 16:02:15.777036 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.776917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm" event={"ID":"d85245f1-c371-40c0-884b-372c86ede2b0","Type":"ContainerDied","Data":"1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9"}
Sep 08 16:02:15.777036 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.776936    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"
Sep 08 16:02:15.777036 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.776976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm" event={"ID":"d85245f1-c371-40c0-884b-372c86ede2b0","Type":"ContainerDied","Data":"85087b4e07496f7efb453af6c4332a0ff670ddd0a19be481e84fbdfed2a12a6b"}
Sep 08 16:02:15.786204 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.786174    2563 scope.go:117] "RemoveContainer" containerID="45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267"
Sep 08 16:02:15.786653 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:02:15.786623    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267\": container with ID starting with 45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267 not found: ID does not exist" containerID="45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267"
Sep 08 16:02:15.786763 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.786663    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267"} err="failed to get container status \"45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267\": rpc error: code = NotFound desc = could not find container \"45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267\": container with ID starting with 45f64ae9640c9f07f3483441ab7af39adf10ec75a3f6ffe6257a9a7ec2794267 not found: ID does not exist"
Sep 08 16:02:15.786763 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.786686    2563 scope.go:117] "RemoveContainer" containerID="1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9"
Sep 08 16:02:15.799562 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.799538    2563 scope.go:117] "RemoveContainer" containerID="1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9"
Sep 08 16:02:15.800199 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:02:15.800120    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9\": container with ID starting with 1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9 not found: ID does not exist" containerID="1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9"
Sep 08 16:02:15.800199 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.800160    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9"} err="failed to get container status \"1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9\": rpc error: code = NotFound desc = could not find container \"1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9\": container with ID starting with 1904d6610aa1538c0feedc35a66c7b4f0e4c3d501d93cc59f302b3ecefe2b4e9 not found: ID does not exist"
Sep 08 16:02:15.803398 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.803369    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"]
Sep 08 16:02:15.805464 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.805439    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b30e99-69b8fb9567-62427"]
Sep 08 16:02:15.815587 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.815549    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"]
Sep 08 16:02:15.819664 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:15.819638    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-b30e99-665cf768fd-cvcdm"]
Sep 08 16:02:16.102783 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:16.102698    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54df6c43-c5af-4f4d-a001-366e586d9b00" path="/var/lib/kubelet/pods/54df6c43-c5af-4f4d-a001-366e586d9b00/volumes"
Sep 08 16:02:16.103010 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:16.102996    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b71772a-7fbf-44c5-8d78-d52a850d1bd6" path="/var/lib/kubelet/pods/8b71772a-7fbf-44c5-8d78-d52a850d1bd6/volumes"
Sep 08 16:02:16.103327 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:16.103277    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d85245f1-c371-40c0-884b-372c86ede2b0" path="/var/lib/kubelet/pods/d85245f1-c371-40c0-884b-372c86ede2b0/volumes"
Sep 08 16:02:21.956489 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:21.956452    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"]
Sep 08 16:02:21.956972 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:21.956662    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" containerID="cri-o://deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e" gracePeriod=30
Sep 08 16:02:26.947209 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:26.947123    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 16:02:26.947209 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:26.947185    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 16:02:27.225030 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.224998    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 16:02:27.356071 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-token\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356252 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-podinfo\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356252 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-data\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356252 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356148    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-envoy\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356252 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-credential-socket\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356252 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356224    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9thlq\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-kube-api-access-9thlq\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356565 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356263    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-certs\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356565 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356332    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24daa250-2f29-470f-8f95-30a6a0d62e6e-istiod-ca-cert\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356565 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-socket\") pod \"24daa250-2f29-470f-8f95-30a6a0d62e6e\" (UID: \"24daa250-2f29-470f-8f95-30a6a0d62e6e\") "
Sep 08 16:02:27.356565 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356534    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:02:27.356774 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356660    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:02:27.356774 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356724    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:02:27.356774 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356739    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-credential-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.357010 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356985    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24daa250-2f29-470f-8f95-30a6a0d62e6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:02:27.357010 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.356998    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:02:27.359030 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.359000    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:02:27.359132 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.359052    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 16:02:27.359132 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.359054    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-kube-api-access-9thlq" (OuterVolumeSpecName: "kube-api-access-9thlq") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "kube-api-access-9thlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:27.359204 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.359131    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "24daa250-2f29-470f-8f95-30a6a0d62e6e" (UID: "24daa250-2f29-470f-8f95-30a6a0d62e6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458192    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-podinfo\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458231    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-data\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458239    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-envoy\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458247    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9thlq\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-kube-api-access-9thlq\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458258    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-certs\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458265    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24daa250-2f29-470f-8f95-30a6a0d62e6e-istiod-ca-cert\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458273    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24daa250-2f29-470f-8f95-30a6a0d62e6e-workload-socket\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.458302 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.458280    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24daa250-2f29-470f-8f95-30a6a0d62e6e-istio-token\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:27.852915 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.852822    2563 generic.go:356] "Generic (PLEG): container finished" podID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerID="deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e" exitCode=0
Sep 08 16:02:27.852915 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.852881    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"
Sep 08 16:02:27.853101 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.852917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" event={"ID":"24daa250-2f29-470f-8f95-30a6a0d62e6e","Type":"ContainerDied","Data":"deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e"}
Sep 08 16:02:27.853101 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.853013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92" event={"ID":"24daa250-2f29-470f-8f95-30a6a0d62e6e","Type":"ContainerDied","Data":"59db92839365d158513856a39621dc0ffa827c0db72ea64bc7f7eaceb1afff47"}
Sep 08 16:02:27.853101 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.853046    2563 scope.go:117] "RemoveContainer" containerID="deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e"
Sep 08 16:02:27.864514 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.864489    2563 scope.go:117] "RemoveContainer" containerID="9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"
Sep 08 16:02:27.874246 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.874208    2563 scope.go:117] "RemoveContainer" containerID="deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e"
Sep 08 16:02:27.874598 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:02:27.874575    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e\": container with ID starting with deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e not found: ID does not exist" containerID="deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e"
Sep 08 16:02:27.874681 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.874610    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e"} err="failed to get container status \"deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e\": rpc error: code = NotFound desc = could not find container \"deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e\": container with ID starting with deb73406787a77b9a9f56cbcdaedd38e22dd0c0edc0d3e5b14abff443006a77e not found: ID does not exist"
Sep 08 16:02:27.874681 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.874631    2563 scope.go:117] "RemoveContainer" containerID="9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"
Sep 08 16:02:27.874836 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:02:27.874818    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f\": container with ID starting with 9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f not found: ID does not exist" containerID="9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"
Sep 08 16:02:27.874888 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.874847    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f"} err="failed to get container status \"9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f\": rpc error: code = NotFound desc = could not find container \"9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f\": container with ID starting with 9ad19ce18f580930f10b13e3f093946797052d4dde5b53fbf2fd84fca08d208f not found: ID does not exist"
Sep 08 16:02:27.875311 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.875273    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"]
Sep 08 16:02:27.880447 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:27.880415    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b30e99-openshift-default-6d4c67cbd-t6z92"]
Sep 08 16:02:28.100707 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:28.100676    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" path="/var/lib/kubelet/pods/24daa250-2f29-470f-8f95-30a6a0d62e6e/volumes"
Sep 08 16:02:32.848459 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:32.848419    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"]
Sep 08 16:02:32.848954 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:32.848729    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd" podUID="9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" containerName="payload-processing" containerID="cri-o://485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690" gracePeriod=30
Sep 08 16:02:32.855583 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:32.855553    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"]
Sep 08 16:02:32.855827 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:32.855794    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb" podUID="0b811684-42e7-4ca4-98ae-ccf0f31d73cf" containerName="payload-pre-processing" containerID="cri-o://6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646" gracePeriod=30
Sep 08 16:02:33.130176 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.130150    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 16:02:33.133034 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.133011    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 16:02:33.320932 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.320901    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5zhw\" (UniqueName: \"kubernetes.io/projected/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-kube-api-access-m5zhw\") pod \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") "
Sep 08 16:02:33.321133 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.320958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-plugins-config-volume\") pod \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\" (UID: \"0b811684-42e7-4ca4-98ae-ccf0f31d73cf\") "
Sep 08 16:02:33.321133 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.320999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-plugins-config-volume\") pod \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") "
Sep 08 16:02:33.321133 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.321020    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwllt\" (UniqueName: \"kubernetes.io/projected/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-kube-api-access-xwllt\") pod \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\" (UID: \"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76\") "
Sep 08 16:02:33.321371 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.321347    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b811684-42e7-4ca4-98ae-ccf0f31d73cf" (UID: "0b811684-42e7-4ca4-98ae-ccf0f31d73cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:02:33.321439 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.321378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" (UID: "9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 16:02:33.323118 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.323088    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-kube-api-access-m5zhw" (OuterVolumeSpecName: "kube-api-access-m5zhw") pod "0b811684-42e7-4ca4-98ae-ccf0f31d73cf" (UID: "0b811684-42e7-4ca4-98ae-ccf0f31d73cf"). InnerVolumeSpecName "kube-api-access-m5zhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:33.323118 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.323087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-kube-api-access-xwllt" (OuterVolumeSpecName: "kube-api-access-xwllt") pod "9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" (UID: "9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76"). InnerVolumeSpecName "kube-api-access-xwllt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:02:33.422217 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.422129    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5zhw\" (UniqueName: \"kubernetes.io/projected/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-kube-api-access-m5zhw\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:33.422217 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.422160    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b811684-42e7-4ca4-98ae-ccf0f31d73cf-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:33.422217 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.422169    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-plugins-config-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:33.422217 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.422178    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwllt\" (UniqueName: \"kubernetes.io/projected/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76-kube-api-access-xwllt\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:02:33.891344 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.891282    2563 generic.go:356] "Generic (PLEG): container finished" podID="9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" containerID="485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690" exitCode=0
Sep 08 16:02:33.891805 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.891350    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"
Sep 08 16:02:33.891805 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.891386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd" event={"ID":"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76","Type":"ContainerDied","Data":"485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690"}
Sep 08 16:02:33.891805 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.891440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd" event={"ID":"9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76","Type":"ContainerDied","Data":"efb397c9c2751d44dcf6427fbd7147f2c9638e95a2648efb15aad598c29722cf"}
Sep 08 16:02:33.891805 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.891466    2563 scope.go:117] "RemoveContainer" containerID="485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690"
Sep 08 16:02:33.892853 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.892778    2563 generic.go:356] "Generic (PLEG): container finished" podID="0b811684-42e7-4ca4-98ae-ccf0f31d73cf" containerID="6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646" exitCode=0
Sep 08 16:02:33.892853 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.892827    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"
Sep 08 16:02:33.892981 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.892866    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb" event={"ID":"0b811684-42e7-4ca4-98ae-ccf0f31d73cf","Type":"ContainerDied","Data":"6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646"}
Sep 08 16:02:33.892981 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.892923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb" event={"ID":"0b811684-42e7-4ca4-98ae-ccf0f31d73cf","Type":"ContainerDied","Data":"d3d4771603440b8a1f1ccd37972b5365234b35fa866771aa6e974b43e7c13dcb"}
Sep 08 16:02:33.908452 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.908430    2563 scope.go:117] "RemoveContainer" containerID="485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690"
Sep 08 16:02:33.908771 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:02:33.908738    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690\": container with ID starting with 485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690 not found: ID does not exist" containerID="485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690"
Sep 08 16:02:33.908838 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.908779    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690"} err="failed to get container status \"485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690\": rpc error: code = NotFound desc = could not find container \"485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690\": container with ID starting with 485ed8cd2fc108b9417aba2d885978b098ce9137deaf46cc60d4bbfbc5d08690 not found: ID does not exist"
Sep 08 16:02:33.908838 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.908797    2563 scope.go:117] "RemoveContainer" containerID="6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646"
Sep 08 16:02:33.918501 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.918473    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"]
Sep 08 16:02:33.919936 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.919917    2563 scope.go:117] "RemoveContainer" containerID="6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646"
Sep 08 16:02:33.921614 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:02:33.921582    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646\": container with ID starting with 6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646 not found: ID does not exist" containerID="6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646"
Sep 08 16:02:33.921741 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.921626    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646"} err="failed to get container status \"6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646\": rpc error: code = NotFound desc = could not find container \"6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646\": container with ID starting with 6037a3e86d22aff8aebc0fda8a31cc39f09dbdcd08ebb3bfe864ed91abaa8646 not found: ID does not exist"
Sep 08 16:02:33.923603 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.923573    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-642d3b-65c8b4d7b7-c6dvd"]
Sep 08 16:02:33.934455 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.934431    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"]
Sep 08 16:02:33.936663 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:33.936643    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-642d3b-6796cc7d58-f7bvb"]
Sep 08 16:02:34.100489 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:34.100455    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b811684-42e7-4ca4-98ae-ccf0f31d73cf" path="/var/lib/kubelet/pods/0b811684-42e7-4ca4-98ae-ccf0f31d73cf/volumes"
Sep 08 16:02:34.100847 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:34.100835    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" path="/var/lib/kubelet/pods/9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76/volumes"
Sep 08 16:02:34.530129 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:02:34.530096    2563 scope.go:117] "RemoveContainer" containerID="648398cfdb16297288ce8209ec3a1afff14c67554a58c5708ede3fa8b539d6fd"
Sep 08 16:03:34.573951 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:03:34.573918    2563 scope.go:117] "RemoveContainer" containerID="54997e4579b57d44b19ca7c2dd6ecffef53da298c261d0d45fc43b6be010586c"
Sep 08 16:03:34.583045 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:03:34.583020    2563 scope.go:117] "RemoveContainer" containerID="88dfa0e0ea0077dd35a62f0285e71bfd13705163f4b12180d49295097d008713"
Sep 08 16:03:34.591467 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:03:34.591446    2563 scope.go:117] "RemoveContainer" containerID="848e30f94c144333078bc597df0d3e8b8b1911696a50e83f1700f97ebafb97c9"
Sep 08 16:05:17.692958 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:17.692920    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"]
Sep 08 16:05:17.693498 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:17.693133    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn" podUID="b2c23c35-15ae-4612-a5b3-c357419f9cd8" containerName="manager" containerID="cri-o://74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285" gracePeriod=10
Sep 08 16:05:19.650799 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.650768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 16:05:19.792200 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.792100    2563 generic.go:356] "Generic (PLEG): container finished" podID="b2c23c35-15ae-4612-a5b3-c357419f9cd8" containerID="74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285" exitCode=0
Sep 08 16:05:19.792200 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.792149    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"
Sep 08 16:05:19.792200 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.792183    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn" event={"ID":"b2c23c35-15ae-4612-a5b3-c357419f9cd8","Type":"ContainerDied","Data":"74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285"}
Sep 08 16:05:19.792442 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.792265    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn" event={"ID":"b2c23c35-15ae-4612-a5b3-c357419f9cd8","Type":"ContainerDied","Data":"9ecc8282b6135b46e2126268fea5a8e5cd16a70c7f852d79ac8e7c23c6cb004c"}
Sep 08 16:05:19.792442 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.792302    2563 scope.go:117] "RemoveContainer" containerID="74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285"
Sep 08 16:05:19.801081 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.801063    2563 scope.go:117] "RemoveContainer" containerID="74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285"
Sep 08 16:05:19.801379 ip-10-0-135-240 kubenswrapper[2563]: E0908 16:05:19.801360    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285\": container with ID starting with 74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285 not found: ID does not exist" containerID="74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285"
Sep 08 16:05:19.801437 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.801390    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285"} err="failed to get container status \"74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285\": rpc error: code = NotFound desc = could not find container \"74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285\": container with ID starting with 74ca45817de66f943ee3bb34b6229ad3958c27d8ba3cb40f545ddf182d2d6285 not found: ID does not exist"
Sep 08 16:05:19.825749 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.825711    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2c23c35-15ae-4612-a5b3-c357419f9cd8-extensions-socket-volume\") pod \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") "
Sep 08 16:05:19.825881 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.825805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9s94\" (UniqueName: \"kubernetes.io/projected/b2c23c35-15ae-4612-a5b3-c357419f9cd8-kube-api-access-p9s94\") pod \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\" (UID: \"b2c23c35-15ae-4612-a5b3-c357419f9cd8\") "
Sep 08 16:05:19.826113 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.826090    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2c23c35-15ae-4612-a5b3-c357419f9cd8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b2c23c35-15ae-4612-a5b3-c357419f9cd8" (UID: "b2c23c35-15ae-4612-a5b3-c357419f9cd8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 16:05:19.827906 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.827875    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2c23c35-15ae-4612-a5b3-c357419f9cd8-kube-api-access-p9s94" (OuterVolumeSpecName: "kube-api-access-p9s94") pod "b2c23c35-15ae-4612-a5b3-c357419f9cd8" (UID: "b2c23c35-15ae-4612-a5b3-c357419f9cd8"). InnerVolumeSpecName "kube-api-access-p9s94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 16:05:19.926935 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.926891    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2c23c35-15ae-4612-a5b3-c357419f9cd8-extensions-socket-volume\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:05:19.926935 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:19.926927    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9s94\" (UniqueName: \"kubernetes.io/projected/b2c23c35-15ae-4612-a5b3-c357419f9cd8-kube-api-access-p9s94\") on node \"ip-10-0-135-240.ec2.internal\" DevicePath \"\""
Sep 08 16:05:20.115204 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:20.115171    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"]
Sep 08 16:05:20.118447 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:20.118415    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pg4dn"]
Sep 08 16:05:22.101110 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:05:22.101075    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2c23c35-15ae-4612-a5b3-c357419f9cd8" path="/var/lib/kubelet/pods/b2c23c35-15ae-4612-a5b3-c357419f9cd8/volumes"
Sep 08 16:06:18.391157 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391117    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"]
Sep 08 16:06:18.391602 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391558    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2c23c35-15ae-4612-a5b3-c357419f9cd8" containerName="manager"
Sep 08 16:06:18.391602 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391571    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2c23c35-15ae-4612-a5b3-c357419f9cd8" containerName="manager"
Sep 08 16:06:18.391602 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391584    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy"
Sep 08 16:06:18.391602 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391589    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy"
Sep 08 16:06:18.391602 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391597    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd86630c-402b-49f4-9227-57ba3e08e1b7" containerName="revoke-keys"
Sep 08 16:06:18.391602 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391602    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd86630c-402b-49f4-9227-57ba3e08e1b7" containerName="revoke-keys"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391617    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391622    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391635    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85245f1-c371-40c0-884b-372c86ede2b0" containerName="payload-pre-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391641    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85245f1-c371-40c0-884b-372c86ede2b0" containerName="payload-pre-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391648    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54df6c43-c5af-4f4d-a001-366e586d9b00" containerName="payload-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391653    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="54df6c43-c5af-4f4d-a001-366e586d9b00" containerName="payload-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391660    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b811684-42e7-4ca4-98ae-ccf0f31d73cf" containerName="payload-pre-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391665    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b811684-42e7-4ca4-98ae-ccf0f31d73cf" containerName="payload-pre-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391670    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" containerName="payload-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391675    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" containerName="payload-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391742    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="54df6c43-c5af-4f4d-a001-366e586d9b00" containerName="payload-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391749    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391755    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b811684-42e7-4ca4-98ae-ccf0f31d73cf" containerName="payload-pre-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391763    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2c23c35-15ae-4612-a5b3-c357419f9cd8" containerName="manager"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391767    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd86630c-402b-49f4-9227-57ba3e08e1b7" containerName="revoke-keys"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391774    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c77a1f9-1f1b-4f52-8255-2f9d80ef7d76" containerName="payload-processing"
Sep 08 16:06:18.391800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.391780    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d85245f1-c371-40c0-884b-372c86ede2b0" containerName="payload-pre-processing"
Sep 08 16:06:18.394916 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.394899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.397363 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.397343    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-pmd7w\""
Sep 08 16:06:18.405929 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.405902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"]
Sep 08 16:06:18.497138 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.497105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62rh8\" (UniqueName: \"kubernetes.io/projected/6f84c41e-8fd5-429e-b981-ac8b011da34d-kube-api-access-62rh8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pc5t2\" (UID: \"6f84c41e-8fd5-429e-b981-ac8b011da34d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.497138 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.497146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f84c41e-8fd5-429e-b981-ac8b011da34d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pc5t2\" (UID: \"6f84c41e-8fd5-429e-b981-ac8b011da34d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.598428 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.598369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62rh8\" (UniqueName: \"kubernetes.io/projected/6f84c41e-8fd5-429e-b981-ac8b011da34d-kube-api-access-62rh8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pc5t2\" (UID: \"6f84c41e-8fd5-429e-b981-ac8b011da34d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.598428 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.598433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f84c41e-8fd5-429e-b981-ac8b011da34d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pc5t2\" (UID: \"6f84c41e-8fd5-429e-b981-ac8b011da34d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.598796 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.598774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f84c41e-8fd5-429e-b981-ac8b011da34d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pc5t2\" (UID: \"6f84c41e-8fd5-429e-b981-ac8b011da34d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.606909 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.606884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62rh8\" (UniqueName: \"kubernetes.io/projected/6f84c41e-8fd5-429e-b981-ac8b011da34d-kube-api-access-62rh8\") pod \"kuadrant-operator-controller-manager-76f8f686bf-pc5t2\" (UID: \"6f84c41e-8fd5-429e-b981-ac8b011da34d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.705843 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.705807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:18.986916 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:18.986823    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"]
Sep 08 16:06:18.990129 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:06:18.990097    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f84c41e_8fd5_429e_b981_ac8b011da34d.slice/crio-707fbfe72c01baff97f7c20fc60cb865b183d1a831dc98f141996586613dea71 WatchSource:0}: Error finding container 707fbfe72c01baff97f7c20fc60cb865b183d1a831dc98f141996586613dea71: Status 404 returned error can't find the container with id 707fbfe72c01baff97f7c20fc60cb865b183d1a831dc98f141996586613dea71
Sep 08 16:06:19.109765 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:19.109717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2" event={"ID":"6f84c41e-8fd5-429e-b981-ac8b011da34d","Type":"ContainerStarted","Data":"7d648fba78c90c7e26ae0106892acb37a74abf65f0289fca7d7be9dc9fdb9803"}
Sep 08 16:06:19.109951 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:19.109774    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2" event={"ID":"6f84c41e-8fd5-429e-b981-ac8b011da34d","Type":"ContainerStarted","Data":"707fbfe72c01baff97f7c20fc60cb865b183d1a831dc98f141996586613dea71"}
Sep 08 16:06:19.109951 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:19.109856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:19.129616 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:19.129562    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2" podStartSLOduration=1.129545411 podStartE2EDuration="1.129545411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 16:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:06:19.127270075 +0000 UTC m=+2025.570462303" watchObservedRunningTime="2026-09-08 16:06:19.129545411 +0000 UTC m=+2025.572737620"
Sep 08 16:06:30.117398 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:30.117358    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-pc5t2"
Sep 08 16:06:34.643599 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:34.643561    2563 scope.go:117] "RemoveContainer" containerID="417a3d9b8c521bf677f685c29b71dceeb91a0ebe4a63600bab222f491fd1cfdb"
Sep 08 16:06:34.652240 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:34.652218    2563 scope.go:117] "RemoveContainer" containerID="236ed4a4cf69882982f515397983e77216306327c7a44ec4feb3685536429dbe"
Sep 08 16:06:34.660368 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:06:34.660341    2563 scope.go:117] "RemoveContainer" containerID="b4515355bb5e22e9b92e7f710c85950f51dc48a205f952b299c2dac24cf2332d"
Sep 08 16:07:34.689430 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:07:34.689394    2563 scope.go:117] "RemoveContainer" containerID="0f6eb0300c7671af63c64a842ee12b19a8a6c42120c452a3e73f95a635d169ef"
Sep 08 16:08:34.711609 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:34.711518    2563 scope.go:117] "RemoveContainer" containerID="68ad5fd6b95ebf75b0941e8488d473caa3e5e969e0c9d6c32e3afc8271e80b82"
Sep 08 16:08:35.804914 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:35.804882    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bll9l_6a31c7af-5184-42fe-a823-7aaae4c2a2ec/manager/0.log"
Sep 08 16:08:36.557756 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:36.557727    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d887cf8-bk6hl_0a69ef28-d232-4778-aabe-c85b4ca4cf2b/copy-manifests/0.log"
Sep 08 16:08:36.563474 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:36.563452    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d887cf8-bk6hl_0a69ef28-d232-4778-aabe-c85b4ca4cf2b/manager/0.log"
Sep 08 16:08:37.283154 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:37.283120    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814705-9qrm5_bcb10c3e-f7a7-4df8-bf9d-b979c5f51c5b/cleanup/0.log"
Sep 08 16:08:37.397783 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:37.397755    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814720-j8nsl_f7163289-ebb4-4905-831a-a0529822f69a/cleanup/0.log"
Sep 08 16:08:37.515232 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:37.515199    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-bf726_dd06e090-eda2-4ddc-9149-5f5920b07847/postgres/0.log"
Sep 08 16:08:38.590006 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:38.589977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-wb4v8_410e00e5-449b-43ad-8370-69fd2f853008/manager/0.log"
Sep 08 16:08:38.832922 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:38.832891    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-76f8f686bf-pc5t2_6f84c41e-8fd5-429e-b981-ac8b011da34d/manager/0.log"
Sep 08 16:08:39.079014 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:39.078977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-9r7kc_1aeecad6-53dc-458a-858c-8793e9eca93a/manager/0.log"
Sep 08 16:08:39.441656 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:39.441625    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4_d5c9c616-f8dd-4a83-bedd-a2a3ddf86609/istio-proxy/0.log"
Sep 08 16:08:39.919771 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:39.919685    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/1.log"
Sep 08 16:08:40.174180 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:40.174152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v54tt_3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878/payload-processing/0.log"
Sep 08 16:08:40.297512 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:40.297476    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77fb65d46d-j8tqb_66e63dad-53d0-4dc5-bb24-421801c5b627/router/0.log"
Sep 08 16:08:40.658257 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:40.658178    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-glxtb_c98a5bad-0721-4837-ae3e-dce0bd31bde5/main/0.log"
Sep 08 16:08:40.901679 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:40.901644    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-w7zc2_7b3fd397-6fb8-4e47-8875-1340948ecc71/main/0.log"
Sep 08 16:08:41.137931 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:41.137901    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ft8mzg_002e5645-c39e-4884-99b3-59f2711005e0/main/0.log"
Sep 08 16:08:50.306862 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:50.306830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pbvp6_13af96a0-8a7f-479c-887d-54f937793c41/global-pull-secret-syncer/0.log"
Sep 08 16:08:50.437750 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:50.435156    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4xdfr_371a2499-4888-48a7-99ea-2307c24cdad6/konnectivity-agent/0.log"
Sep 08 16:08:50.524225 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:50.524197    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-240.ec2.internal_27659e071ebcf76aabb813758c1a03eb/haproxy/0.log"
Sep 08 16:08:55.056962 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:55.056929    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-wb4v8_410e00e5-449b-43ad-8370-69fd2f853008/manager/0.log"
Sep 08 16:08:55.128939 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:55.128907    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-76f8f686bf-pc5t2_6f84c41e-8fd5-429e-b981-ac8b011da34d/manager/0.log"
Sep 08 16:08:55.207389 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:55.207355    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-9r7kc_1aeecad6-53dc-458a-858c-8793e9eca93a/manager/0.log"
Sep 08 16:08:56.739668 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.739635    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/alertmanager/0.log"
Sep 08 16:08:56.761566 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.761517    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/config-reloader/0.log"
Sep 08 16:08:56.782654 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.782626    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/kube-rbac-proxy-web/0.log"
Sep 08 16:08:56.807075 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.807046    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/kube-rbac-proxy/0.log"
Sep 08 16:08:56.837431 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.837401    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/kube-rbac-proxy-metric/0.log"
Sep 08 16:08:56.863086 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.863055    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/prom-label-proxy/0.log"
Sep 08 16:08:56.885081 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.884986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d7da9185-fcba-46dd-aad2-a7b588bfbde7/init-config-reloader/0.log"
Sep 08 16:08:56.918900 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:56.918866    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-zxhhj_d99c67bb-3c5e-41fb-84af-6d4441c8ea78/cluster-monitoring-operator/0.log"
Sep 08 16:08:57.044683 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.044641    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-pjm68_72959bd5-6be2-4c81-a585-02777b50fff6/monitoring-plugin/0.log"
Sep 08 16:08:57.074139 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.074111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gsdjv_9173d83b-f216-4aed-9c2f-e7e19f9c86e3/node-exporter/0.log"
Sep 08 16:08:57.098913 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.098889    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gsdjv_9173d83b-f216-4aed-9c2f-e7e19f9c86e3/kube-rbac-proxy/0.log"
Sep 08 16:08:57.121708 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.121683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gsdjv_9173d83b-f216-4aed-9c2f-e7e19f9c86e3/init-textfile/0.log"
Sep 08 16:08:57.406072 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.406035    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/prometheus/0.log"
Sep 08 16:08:57.437480 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.437458    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/config-reloader/0.log"
Sep 08 16:08:57.466523 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.466500    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/thanos-sidecar/0.log"
Sep 08 16:08:57.500800 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.500774    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/kube-rbac-proxy-web/0.log"
Sep 08 16:08:57.529010 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.528984    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/kube-rbac-proxy/0.log"
Sep 08 16:08:57.560701 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.560674    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/kube-rbac-proxy-thanos/0.log"
Sep 08 16:08:57.585471 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.585450    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7edfe10f-82b1-4f43-b3f7-c6790651dd6d/init-config-reloader/0.log"
Sep 08 16:08:57.615531 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.615507    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-rmbj5_57aa1c97-2776-4750-9282-08498e43ae48/prometheus-operator/0.log"
Sep 08 16:08:57.640652 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.640622    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-rmbj5_57aa1c97-2776-4750-9282-08498e43ae48/kube-rbac-proxy/0.log"
Sep 08 16:08:57.666802 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.666772    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-2hg4r_bcdeb6e7-d979-4470-90e5-62fe8324dd90/prometheus-operator-admission-webhook/0.log"
Sep 08 16:08:57.702892 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.702865    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b95879bfd-mn6hd_c1e87551-aeea-4cb2-9473-98f5c1491e9e/telemeter-client/0.log"
Sep 08 16:08:57.727965 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.727936    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b95879bfd-mn6hd_c1e87551-aeea-4cb2-9473-98f5c1491e9e/reload/0.log"
Sep 08 16:08:57.748206 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:57.748173    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b95879bfd-mn6hd_c1e87551-aeea-4cb2-9473-98f5c1491e9e/kube-rbac-proxy/0.log"
Sep 08 16:08:59.412335 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:08:59.412278    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-pkwt5_09eee509-c68d-4c0c-ab20-67f5589a9f23/networking-console-plugin/0.log"
Sep 08 16:09:01.226413 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.226378    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"]
Sep 08 16:09:01.226846 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.226833    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="24daa250-2f29-470f-8f95-30a6a0d62e6e" containerName="istio-proxy"
Sep 08 16:09:01.230385 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.230363    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.232670 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.232646    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zhg9q\"/\"kube-root-ca.crt\""
Sep 08 16:09:01.233424 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.233406    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zhg9q\"/\"default-dockercfg-7qzb5\""
Sep 08 16:09:01.233554 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.233426    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zhg9q\"/\"openshift-service-ca.crt\""
Sep 08 16:09:01.241202 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.241178    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"]
Sep 08 16:09:01.311949 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.311903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-proc\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.312145 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.312024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-sys\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.312145 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.312057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-lib-modules\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.312145 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.312108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb8l9\" (UniqueName: \"kubernetes.io/projected/9868bbaa-bcce-4429-90b6-f4457307493b-kube-api-access-cb8l9\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.312278 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.312168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-podres\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413352 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-sys\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413352 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-lib-modules\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413625 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-sys\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413625 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb8l9\" (UniqueName: \"kubernetes.io/projected/9868bbaa-bcce-4429-90b6-f4457307493b-kube-api-access-cb8l9\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413625 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-podres\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413625 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-proc\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413625 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-lib-modules\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413625 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-proc\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.413844 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.413642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9868bbaa-bcce-4429-90b6-f4457307493b-podres\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.421568 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.421535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb8l9\" (UniqueName: \"kubernetes.io/projected/9868bbaa-bcce-4429-90b6-f4457307493b-kube-api-access-cb8l9\") pod \"perf-node-gather-daemonset-npfvk\" (UID: \"9868bbaa-bcce-4429-90b6-f4457307493b\") " pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.540679 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.540587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.813247 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.813218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"]
Sep 08 16:09:01.815965 ip-10-0-135-240 kubenswrapper[2563]: W0908 16:09:01.815940    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9868bbaa_bcce_4429_90b6_f4457307493b.slice/crio-4b6f5bb35502d931d7aed465cf026d1f809bab914666f23619810e28ee3fd07f WatchSource:0}: Error finding container 4b6f5bb35502d931d7aed465cf026d1f809bab914666f23619810e28ee3fd07f: Status 404 returned error can't find the container with id 4b6f5bb35502d931d7aed465cf026d1f809bab914666f23619810e28ee3fd07f
Sep 08 16:09:01.974031 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.973996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk" event={"ID":"9868bbaa-bcce-4429-90b6-f4457307493b","Type":"ContainerStarted","Data":"aafcdf884464796efb5f430da0876d84eff030e97381368068f05006d9143ebc"}
Sep 08 16:09:01.974209 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.974063    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:01.974209 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.974081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk" event={"ID":"9868bbaa-bcce-4429-90b6-f4457307493b","Type":"ContainerStarted","Data":"4b6f5bb35502d931d7aed465cf026d1f809bab914666f23619810e28ee3fd07f"}
Sep 08 16:09:01.989888 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:01.989768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk" podStartSLOduration=0.989749093 podStartE2EDuration="989.749093ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 16:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 16:09:01.9895003 +0000 UTC m=+2188.432692519" watchObservedRunningTime="2026-09-08 16:09:01.989749093 +0000 UTC m=+2188.432941302"
Sep 08 16:09:02.549070 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:02.549032    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvvx9_706badb4-d406-4b28-8a58-8128f1a97764/dns/0.log"
Sep 08 16:09:02.572801 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:02.572768    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvvx9_706badb4-d406-4b28-8a58-8128f1a97764/kube-rbac-proxy/0.log"
Sep 08 16:09:02.623061 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:02.623023    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qldnr_a82652f7-3dc5-4132-9dd6-02e12e6a95cd/dns-node-resolver/0.log"
Sep 08 16:09:03.237422 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:03.237391    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v2cz2_3f439e87-2fcb-430d-960b-eb74a8e5efdd/node-ca/0.log"
Sep 08 16:09:04.149487 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:04.149455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4_d5c9c616-f8dd-4a83-bedd-a2a3ddf86609/istio-proxy/0.log"
Sep 08 16:09:04.785922 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:04.785888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/0.log"
Sep 08 16:09:04.795484 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:04.795461    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/1.log"
Sep 08 16:09:04.877350 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:04.877308    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v54tt_3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878/payload-processing/0.log"
Sep 08 16:09:04.902046 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:04.902013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77fb65d46d-j8tqb_66e63dad-53d0-4dc5-bb24-421801c5b627/router/0.log"
Sep 08 16:09:05.733186 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:05.733161    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9bk86_65240601-6812-40d6-ba6f-1ef503d4f8ad/serve-healthcheck-canary/0.log"
Sep 08 16:09:06.414106 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:06.414076    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shh8w_135ab67e-b78a-4307-95db-4dea8480e412/kube-rbac-proxy/0.log"
Sep 08 16:09:06.434607 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:06.434582    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shh8w_135ab67e-b78a-4307-95db-4dea8480e412/exporter/0.log"
Sep 08 16:09:06.453963 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:06.453939    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shh8w_135ab67e-b78a-4307-95db-4dea8480e412/extractor/0.log"
Sep 08 16:09:07.989407 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:07.989379    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zhg9q/perf-node-gather-daemonset-npfvk"
Sep 08 16:09:08.538551 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:08.538524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bll9l_6a31c7af-5184-42fe-a823-7aaae4c2a2ec/manager/0.log"
Sep 08 16:09:09.301685 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:09.301650    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d887cf8-bk6hl_0a69ef28-d232-4778-aabe-c85b4ca4cf2b/manager/0.log"
Sep 08 16:09:09.323229 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:09.323199    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84d887cf8-bk6hl_0a69ef28-d232-4778-aabe-c85b4ca4cf2b/copy-manifests/0.log"
Sep 08 16:09:10.729539 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:10.729510    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66d9b677c7-r5ttk_2b0d25fa-aaed-4eb1-b326-28ff04552700/manager/0.log"
Sep 08 16:09:11.081006 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:11.080967    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4_d5c9c616-f8dd-4a83-bedd-a2a3ddf86609/istio-proxy/0.log"
Sep 08 16:09:11.661329 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:11.661227    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/0.log"
Sep 08 16:09:11.671110 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:11.671080    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/1.log"
Sep 08 16:09:11.736660 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:11.736625    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v54tt_3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878/payload-processing/0.log"
Sep 08 16:09:11.750781 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:11.750746    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77fb65d46d-j8tqb_66e63dad-53d0-4dc5-bb24-421801c5b627/router/0.log"
Sep 08 16:09:13.332878 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:13.332794    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-h78r4_d5c9c616-f8dd-4a83-bedd-a2a3ddf86609/istio-proxy/0.log"
Sep 08 16:09:13.920655 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:13.920566    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/0.log"
Sep 08 16:09:13.929726 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:13.929701    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-xprr6_18cb8a11-7ae1-4a8b-a2d3-0e0b084b5bca/istio-proxy/1.log"
Sep 08 16:09:13.994818 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:13.994786    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v54tt_3c3b07f5-bbb4-49f5-9e06-2f91d8ba8878/payload-processing/0.log"
Sep 08 16:09:14.010841 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:14.010808    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77fb65d46d-j8tqb_66e63dad-53d0-4dc5-bb24-421801c5b627/router/0.log"
Sep 08 16:09:17.237410 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.237378    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/kube-multus-additional-cni-plugins/0.log"
Sep 08 16:09:17.260942 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.260908    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/egress-router-binary-copy/0.log"
Sep 08 16:09:17.281754 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.281730    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/cni-plugins/0.log"
Sep 08 16:09:17.303215 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.303190    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/bond-cni-plugin/0.log"
Sep 08 16:09:17.322942 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.322907    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/routeoverride-cni/0.log"
Sep 08 16:09:17.344406 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.344315    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/whereabouts-cni-bincopy/0.log"
Sep 08 16:09:17.367226 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.367198    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxdzc_847c4a7e-f4af-460b-9747-b082b1b323ec/whereabouts-cni/0.log"
Sep 08 16:09:17.933915 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:17.933879    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s9wdl_3de883f2-c452-4202-b140-874f16695b98/kube-multus/0.log"
Sep 08 16:09:18.006159 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:18.006130    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dxkzl_3bdb3868-cedc-47d0-9c6c-cadb57a2b95e/network-metrics-daemon/0.log"
Sep 08 16:09:18.023469 ip-10-0-135-240 kubenswrapper[2563]: I0908 16:09:18.023440    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dxkzl_3bdb3868-cedc-47d0-9c6c-cadb57a2b95e/kube-rbac-proxy/0.log"