Aug 04 15:08:53.168878 ip-10-0-136-40 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Aug 04 15:08:53.168895 ip-10-0-136-40 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Aug 04 15:08:53.168905 ip-10-0-136-40 systemd[1]: kubelet.service: Failed with result 'resources'.
Aug 04 15:08:53.169301 ip-10-0-136-40 systemd[1]: Failed to start Kubernetes Kubelet.
Aug 04 15:09:03.219313 ip-10-0-136-40 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Aug 04 15:09:03.219331 ip-10-0-136-40 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a38f41669eb941c0a402c8e7ed82eed8 --
Aug 04 15:10:57.395977 ip-10-0-136-40 systemd[1]: Starting Kubernetes Kubelet...
Aug 04 15:10:57.720278 ip-10-0-136-40 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 15:10:57.720278 ip-10-0-136-40 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 04 15:10:57.720278 ip-10-0-136-40 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 15:10:57.720278 ip-10-0-136-40 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 04 15:10:57.720278 ip-10-0-136-40 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 04 15:10:57.722086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.721988    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 04 15:10:57.727376 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727352    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 15:10:57.727376 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727372    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 15:10:57.727376 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727377    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 15:10:57.727376 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727383    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 15:10:57.727376 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727387    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727392    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727396    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727400    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727403    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727407    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727411    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727415    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727418    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727422    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727425    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727429    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727433    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727439    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727443    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727447    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727450    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727454    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727458    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 15:10:57.727710 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727464    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727470    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727474    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727479    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727483    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727488    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727494    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727500    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727505    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727510    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727514    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727518    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727522    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727526    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727530    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727535    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727540    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727545    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727549    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727554    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 15:10:57.728413 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727558    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727563    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727566    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727571    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727575    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727579    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727583    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727588    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727592    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727597    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727601    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727606    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727610    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727614    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727618    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727622    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727627    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727631    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727651    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727656    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 15:10:57.729145 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727660    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727664    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727668    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727672    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727676    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727685    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727691    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727696    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727700    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727705    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727709    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727713    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727717    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727722    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727726    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727730    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727734    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727739    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727743    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727748    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 15:10:57.729745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727754    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727758    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727763    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727767    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727771    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727775    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727779    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727783    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727787    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727792    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727795    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727800    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727804    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727808    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727812    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.727817    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730026    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730038    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730042    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730045    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 15:10:57.730230 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730048    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730052    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730055    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730058    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730061    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730064    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730067    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730070    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730072    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730075    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730078    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730081    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730084    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730086    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730089    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730091    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730094    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730096    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730098    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730102    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 15:10:57.730734 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730106    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730109    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730111    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730114    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730117    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730120    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730122    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730125    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730127    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730130    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730134    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730136    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730139    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730142    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730146    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730150    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730153    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730156    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730159    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730162    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 15:10:57.731236 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730164    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730167    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730170    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730172    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730175    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730178    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730181    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730184    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730187    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730189    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730192    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730195    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730197    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730200    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730202    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730205    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730208    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730211    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730213    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 15:10:57.731772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730216    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730218    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730221    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730224    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730227    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730229    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730233    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730237    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730241    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730244    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730247    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730250    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730252    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730257    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730261    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730263    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730266    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730268    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730272    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 15:10:57.732234 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730275    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730279    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730281    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730284    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730287    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730290    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730293    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730295    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730298    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730300    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730303    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730305    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730308    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730310    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730313    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730316    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730319    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730740    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730745    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730748    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 15:10:57.732719 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730751    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730754    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730756    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730759    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730762    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730765    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730768    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730770    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730774    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730778    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730781    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730783    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730787    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730792    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730794    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730797    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730800    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730802    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730805    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730807    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 15:10:57.733295 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730811    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730815    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730819    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730822    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730825    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730828    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730831    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730833    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730836    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730839    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730841    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730844    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730847    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730850    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730852    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730855    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730858    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730861    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730863    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 15:10:57.733822 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730866    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730868    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730871    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730874    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730877    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730879    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730882    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730885    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730888    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730890    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730893    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730896    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730898    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730901    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730903    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730906    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730909    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730912    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730914    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730917    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 15:10:57.734297 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730920    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730922    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730925    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730928    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730930    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730933    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730935    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730937    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730940    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730943    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730946    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730948    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730951    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730953    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730956    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730958    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730960    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730963    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730965    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730968    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 15:10:57.734849 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730971    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730974    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730977    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730979    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730982    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730984    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730987    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730990    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730993    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.730996    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731000    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731003    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731006    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731009    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731012    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731015    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731017    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731099    2569 flags.go:64] FLAG: --address="0.0.0.0"
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731120    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731128    2569 flags.go:64] FLAG: --anonymous-auth="true"
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731133    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 04 15:10:57.735344 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731137    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731140    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731145    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731150    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731153    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731157    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731160    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731164    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731167    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731171    2569 flags.go:64] FLAG: --cgroup-root=""
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731174    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731176    2569 flags.go:64] FLAG: --client-ca-file=""
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731180    2569 flags.go:64] FLAG: --cloud-provider="external"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731183    2569 flags.go:64] FLAG: --cluster-dns="[]"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731187    2569 flags.go:64] FLAG: --cluster-domain=""
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731190    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731194    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731197    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731201    2569 flags.go:64] FLAG: --container-log-max-files="5"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731211    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731214    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731217    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731221    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731224    2569 flags.go:64] FLAG: --contention-profiling="false"
Aug 04 15:10:57.735882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731227    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731230    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731233    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731236    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731241    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731244    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731247    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731250    2569 flags.go:64] FLAG: --enable-load-reader="false"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731253    2569 flags.go:64] FLAG: --enable-server="true"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731256    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731270    2569 flags.go:64] FLAG: --event-burst="100"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731275    2569 flags.go:64] FLAG: --event-qps="50"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731278    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731281    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731284    2569 flags.go:64] FLAG: --eviction-hard=""
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731288    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731291    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731294    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731297    2569 flags.go:64] FLAG: --eviction-soft=""
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731300    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731303    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731307    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731311    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731314    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731319    2569 flags.go:64] FLAG: --fail-swap-on="true"
Aug 04 15:10:57.736467 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731322    2569 flags.go:64] FLAG: --feature-gates=""
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731327    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731330    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731333    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731336    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731340    2569 flags.go:64] FLAG: --healthz-port="10248"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731343    2569 flags.go:64] FLAG: --help="false"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731346    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-136-40.ec2.internal"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731349    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731352    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731355    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731358    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731361    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731365    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731367    2569 flags.go:64] FLAG: --image-service-endpoint=""
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731370    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731373    2569 flags.go:64] FLAG: --kube-api-burst="100"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731376    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731379    2569 flags.go:64] FLAG: --kube-api-qps="50"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731382    2569 flags.go:64] FLAG: --kube-reserved=""
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731385    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731388    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731391    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731394    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 04 15:10:57.737117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731396    2569 flags.go:64] FLAG: --lock-file=""
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731399    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731402    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731405    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731411    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731414    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731417    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731422    2569 flags.go:64] FLAG: --logging-format="text"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731425    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731429    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731432    2569 flags.go:64] FLAG: --manifest-url=""
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731435    2569 flags.go:64] FLAG: --manifest-url-header=""
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731440    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731443    2569 flags.go:64] FLAG: --max-open-files="1000000"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731449    2569 flags.go:64] FLAG: --max-pods="110"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731453    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731458    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731461    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731466    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731471    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731475    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731478    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731486    2569 flags.go:64] FLAG: --node-status-max-images="50"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731489    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731492    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 04 15:10:57.737969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731495    2569 flags.go:64] FLAG: --pod-cidr=""
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731499    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731506    2569 flags.go:64] FLAG: --pod-manifest-path=""
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731508    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731512    2569 flags.go:64] FLAG: --pods-per-core="0"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731514    2569 flags.go:64] FLAG: --port="10250"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731517    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731520    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09317ff48cc43d0b2"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731524    2569 flags.go:64] FLAG: --qos-reserved=""
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731526    2569 flags.go:64] FLAG: --read-only-port="10255"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731530    2569 flags.go:64] FLAG: --register-node="true"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731532    2569 flags.go:64] FLAG: --register-with-taints=""
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731536    2569 flags.go:64] FLAG: --registry-burst="10"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731539    2569 flags.go:64] FLAG: --registry-qps="5"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731544    2569 flags.go:64] FLAG: --reserved-cpus=""
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731547    2569 flags.go:64] FLAG: --reserved-memory=""
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731551    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731554    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731557    2569 flags.go:64] FLAG: --rotate-certificates="false"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731560    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731563    2569 flags.go:64] FLAG: --runonce="false"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731566    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731569    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731572    2569 flags.go:64] FLAG: --seccomp-default="false"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731575    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 04 15:10:57.739098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731578    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731581    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731584    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731587    2569 flags.go:64] FLAG: --storage-driver-password="root"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731590    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731593    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731595    2569 flags.go:64] FLAG: --storage-driver-user="root"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731598    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731601    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731603    2569 flags.go:64] FLAG: --system-cgroups=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731606    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731611    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731614    2569 flags.go:64] FLAG: --tls-cert-file=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731617    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731621    2569 flags.go:64] FLAG: --tls-min-version=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731624    2569 flags.go:64] FLAG: --tls-private-key-file=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731627    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731630    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731633    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731650    2569 flags.go:64] FLAG: --v="2"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731654    2569 flags.go:64] FLAG: --version="false"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731664    2569 flags.go:64] FLAG: --vmodule=""
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731669    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.731672    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731798    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 15:10:57.740082 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731802    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731805    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731808    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731812    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731816    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731819    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731823    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731826    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731828    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731831    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731834    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731836    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731839    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731842    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731844    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731847    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731849    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731852    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731855    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 15:10:57.740889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731858    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731860    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731863    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731865    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731868    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731871    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731873    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731876    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731878    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731882    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731887    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731889    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731892    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731895    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731897    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731900    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731903    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731905    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731908    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731910    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 15:10:57.741742 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731913    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731916    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731918    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731921    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731923    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731926    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731929    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731931    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731934    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731937    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731939    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731942    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731944    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731947    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731949    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731952    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731955    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731957    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731960    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731962    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 15:10:57.742404 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731965    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731967    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731972    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731975    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731978    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731981    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731983    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731986    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731989    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731992    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731994    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.731997    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732000    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732004    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732007    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732011    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732014    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732016    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732019    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 15:10:57.743280 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732023    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732027    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732030    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732032    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732035    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732038    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732040    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732043    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732046    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732048    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732050    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732053    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732056    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732058    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732061    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732065    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732067    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732071    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732073    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 15:10:57.744108 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.732076    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.732085    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.739840    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.739863    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.740502    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.740518    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740614    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740624    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740629    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740652    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740658    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740662    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740667    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740671    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740676    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 15:10:57.744591 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740680    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740685    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740689    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740693    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740697    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740701    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740705    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740709    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740714    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740718    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740722    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740726    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740731    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740735    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740739    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740743    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740748    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740752    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740756    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740760    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 15:10:57.745111 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740768    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740773    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740777    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740781    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740785    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740789    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740794    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740798    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740802    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740807    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740811    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740815    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740819    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740824    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740828    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740832    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740837    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740841    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740845    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740849    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 15:10:57.745623 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740853    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740858    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740862    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740866    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740871    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740875    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740879    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740884    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740888    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740893    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740897    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740901    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740906    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740911    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740915    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740919    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740924    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740928    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740933    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740937    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 15:10:57.746263 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740942    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740946    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740950    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740954    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740958    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740962    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740967    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740971    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740978    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740984    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740989    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740993    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.740998    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741002    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741009    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741015    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741020    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741025    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741030    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 15:10:57.746821 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741037    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741042    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741046    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741052    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741057    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741063    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741067    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741072    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741077    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741082    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741087    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.741096    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741294    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741305    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741309    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 04 15:10:57.747363 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741314    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741319    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741323    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741327    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741331    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741336    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741340    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741346    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741350    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741355    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741359    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741363    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741367    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741371    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741375    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741379    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741383    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741387    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741392    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741396    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 04 15:10:57.747792 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741400    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741405    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741409    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741413    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741417    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741422    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741426    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741431    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741436    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741440    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741444    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741449    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741453    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741457    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741461    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741465    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741469    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741473    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741477    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 04 15:10:57.748466 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741482    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741486    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741490    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741496    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741502    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741506    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741511    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741515    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741519    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741523    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741527    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741532    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741536    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741540    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741545    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741549    2569 feature_gate.go:328] unrecognized feature gate: Example2
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741553    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741557    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741562    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 04 15:10:57.749218 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741567    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741571    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741575    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741580    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741584    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741590    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741596    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741601    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741606    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741611    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741616    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741621    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741626    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741630    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741650    2569 feature_gate.go:328] unrecognized feature gate: Example
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741655    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741659    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741663    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741668    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741673    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 04 15:10:57.749985 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741677    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741682    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741686    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741690    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741697    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741701    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741705    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741709    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741716    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741721    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741726    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741731    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741736    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741741    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741746    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741750    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741754    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 04 15:10:57.750477 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:57.741758    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.741765    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.741968    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.744675    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.746049    2569 server.go:1019] "Starting client certificate rotation"
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.746160    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.746811    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 04 15:10:57.750971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.748711    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 15:10:57.752284 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.752260    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 04 15:10:57.766732 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.766694    2569 log.go:25] "Validated CRI v1 runtime API"
Aug 04 15:10:57.769291 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.769274    2569 log.go:25] "Validated CRI v1 image API"
Aug 04 15:10:57.770251 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.770234    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 04 15:10:57.773375 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.773352    2569 fs.go:135] Filesystem UUIDs: map[35ed441a-821e-43e4-9499-1bb8a1a32d94:/dev/nvme0n1p4 476fe08e-9bf4-4429-aead-1230877466e5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Aug 04 15:10:57.773431 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.773375    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 04 15:10:57.778931 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.778794    2569 manager.go:216] Machine: {Timestamp:2026-08-04 15:10:57.777523226 +0000 UTC m=+0.283821276 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089732 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28cffbf81caf5726082cd310a3b31d SystemUUID:ec28cffb-f81c-af57-2608-2cd310a3b31d BootID:a38f4166-9eb9-41c0-a402-c8e7ed82eed8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f5:d9:6e:72:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f5:d9:6e:72:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:e3:e3:55:3f:e7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 04 15:10:57.778931 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.778916    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 04 15:10:57.779071 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.779016    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 04 15:10:57.782010 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.781979    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 04 15:10:57.782183 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.782012    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 04 15:10:57.782764 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.782752    2569 topology_manager.go:138] "Creating topology manager with none policy"
Aug 04 15:10:57.782806 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.782769    2569 container_manager_linux.go:309] "Creating device plugin manager"
Aug 04 15:10:57.782806 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.782778    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 15:10:57.783435 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.783423    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 04 15:10:57.784393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.784382    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 04 15:10:57.789017 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789002    2569 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 15:10:57.789165 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789154    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 04 15:10:57.789204 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789189    2569 kubelet.go:486] "Attempting to sync node with API server"
Aug 04 15:10:57.789204 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789201    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 04 15:10:57.789263 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789220    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 04 15:10:57.789263 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789230    2569 kubelet.go:398] "Adding apiserver pod source"
Aug 04 15:10:57.789263 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789240    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 04 15:10:57.789656 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789641    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 15:10:57.789740 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.789667    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 04 15:10:57.791727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.791699    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 04 15:10:57.792042 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.792022    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 15:10:57.793175 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.793161    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 04 15:10:57.793229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.793194    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 04 15:10:57.794342 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794330    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 04 15:10:57.794392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794349    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 04 15:10:57.794392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794357    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 04 15:10:57.794392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794364    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 04 15:10:57.794392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794373    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 04 15:10:57.794392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794386    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 04 15:10:57.794536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794395    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 04 15:10:57.794536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794401    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 04 15:10:57.794536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794409    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 04 15:10:57.794536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794415    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 04 15:10:57.794536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794430    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 04 15:10:57.794536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.794439    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 04 15:10:57.795114 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.795103    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 04 15:10:57.795164 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.795116    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 04 15:10:57.798890 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.798873    2569 server.go:1268] "Started kubelet"
Aug 04 15:10:57.799029 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.798970    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 04 15:10:57.799133 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.799110    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 04 15:10:57.799651 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.799117    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 04 15:10:57.800099 ip-10-0-136-40 systemd[1]: Started Kubernetes Kubelet.
Aug 04 15:10:57.802099 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.802069    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 04 15:10:57.802362 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.802334    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 04 15:10:57.802640 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.802611    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 04 15:10:57.802769 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.802722    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 04 15:10:57.804059 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.804036    2569 server.go:310] "Adding debug handlers to kubelet server"
Aug 04 15:10:57.805780 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.805758    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 04 15:10:57.807359 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.806520    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18c8a2cdfa022cfb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-08-04 15:10:57.798843643 +0000 UTC m=+0.305141692,LastTimestamp:2026-08-04 15:10:57.798843643 +0000 UTC m=+0.305141692,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Aug 04 15:10:57.809890 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.809865    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 04 15:10:57.809890 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.809870    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 04 15:10:57.810032 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.809953    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:57.810440 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.810418    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 04 15:10:57.810440 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.810422    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 04 15:10:57.810578 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.810448    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 04 15:10:57.810631 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.810580    2569 reconstruct.go:97] "Volume reconstruction finished"
Aug 04 15:10:57.810631 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.810588    2569 reconciler.go:29] "Reconciler: start to sync state"
Aug 04 15:10:57.812341 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.812261    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 04 15:10:57.812457 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.812365    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 04 15:10:57.812630 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.812608    2569 factory.go:55] Registering systemd factory
Aug 04 15:10:57.812630 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.812633    2569 factory.go:223] Registration of the systemd container factory successfully
Aug 04 15:10:57.812960 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.812942    2569 factory.go:153] Registering CRI-O factory
Aug 04 15:10:57.812960 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.812960    2569 factory.go:223] Registration of the crio container factory successfully
Aug 04 15:10:57.813112 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.813044    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 04 15:10:57.813112 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.813071    2569 factory.go:103] Registering Raw factory
Aug 04 15:10:57.813112 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.813088    2569 manager.go:1196] Started watching for new ooms in manager
Aug 04 15:10:57.813513 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.813499    2569 manager.go:349] Starting recovery of all containers
Aug 04 15:10:57.824984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.824835    2569 manager.go:354] Recovery completed
Aug 04 15:10:57.829314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.829300    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:57.831630 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.831608    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:57.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.831645    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:57.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.831655    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:57.832425 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.832404    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 04 15:10:57.832425 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.832423    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 04 15:10:57.832562 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.832430    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:57.832562 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.832444    2569 state_mem.go:36] "Initialized new in-memory state store"
Aug 04 15:10:57.833253 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.833235    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:57.833320 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.833263    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:57.833320 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.833274    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:57.834678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.834664    2569 policy_none.go:49] "None policy: Start"
Aug 04 15:10:57.834762 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.834681    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 04 15:10:57.834762 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.834693    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 04 15:10:57.836804 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.836791    2569 policy_none.go:47] "Start"
Aug 04 15:10:57.840680 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.840659    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zzz99"
Aug 04 15:10:57.847183 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.847100    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18c8a2cdfbf674f5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-08-04 15:10:57.831630069 +0000 UTC m=+0.337928118,LastTimestamp:2026-08-04 15:10:57.831630069 +0000 UTC m=+0.337928118,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Aug 04 15:10:57.852597 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.852517    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18c8a2cdfbf6c3da  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-40.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-08-04 15:10:57.831650266 +0000 UTC m=+0.337948316,LastTimestamp:2026-08-04 15:10:57.831650266 +0000 UTC m=+0.337948316,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Aug 04 15:10:57.863931 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.863850    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18c8a2cdfbf6e8da  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-40.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-08-04 15:10:57.831659738 +0000 UTC m=+0.337957788,LastTimestamp:2026-08-04 15:10:57.831659738 +0000 UTC m=+0.337957788,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Aug 04 15:10:57.864515 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.864486    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zzz99"
Aug 04 15:10:57.873045 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.872959    2569 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-40.ec2.internal.18c8a2cdfbf674f5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18c8a2cdfbf674f5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-08-04 15:10:57.831630069 +0000 UTC m=+0.337928118,LastTimestamp:2026-08-04 15:10:57.833250808 +0000 UTC m=+0.339548857,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Aug 04 15:10:57.879464 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.879435    2569 manager.go:337] "Starting Device Plugin manager"
Aug 04 15:10:57.879594 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.879480    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 04 15:10:57.879594 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.879493    2569 server.go:87] "Starting device plugin registration server"
Aug 04 15:10:57.879898 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.879883    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 04 15:10:57.879941 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.879902    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 04 15:10:57.880020 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.880000    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 04 15:10:57.880131 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.880112    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 04 15:10:57.880131 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.880128    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 04 15:10:57.880787 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.880767    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 04 15:10:57.880879 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.880822    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:57.957181 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.957130    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 04 15:10:57.959470 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.958459    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 04 15:10:57.959470 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.958512    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 04 15:10:57.959470 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.958540    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 04 15:10:57.959470 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.958587    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 04 15:10:57.961776 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.961755    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 15:10:57.980612 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.980516    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:57.981857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.981839    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:57.981951 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.981874    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:57.981951 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.981886    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:57.981951 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.981914    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-40.ec2.internal"
Aug 04 15:10:57.994798 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:57.994771    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-40.ec2.internal"
Aug 04 15:10:57.994875 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:57.994806    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-40.ec2.internal\": node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.021668 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.021634    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.058787 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.058748    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"]
Aug 04 15:10:58.058872 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.058836    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:58.060531 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.060514    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:58.060615 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.060554    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:58.060615 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.060570    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:58.063101 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.063087    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:58.063205 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.063190    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.063253 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.063225    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:58.064049 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.064027    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:58.064178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.064066    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:58.064178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.064068    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:58.064178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.064081    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:58.064178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.064090    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:58.064178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.064101    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:58.066488 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.066471    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.066560 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.066508    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:58.067351 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.067335    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:58.067423 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.067363    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:58.067423 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.067374    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:58.091863 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.091835    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-40.ec2.internal\" not found" node="ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.096767 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.096744    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-40.ec2.internal\" not found" node="ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.112421 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.112392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/21665e5cbddb8060242764c579d30712-config\") pod \"kube-apiserver-proxy-ip-10-0-136-40.ec2.internal\" (UID: \"21665e5cbddb8060242764c579d30712\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.112553 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.112429    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ceb8cb30e7701f771e3116e5ccd9f414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"ceb8cb30e7701f771e3116e5ccd9f414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.112553 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.112464    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ceb8cb30e7701f771e3116e5ccd9f414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"ceb8cb30e7701f771e3116e5ccd9f414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.122861 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.122808    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.213064 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.213031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ceb8cb30e7701f771e3116e5ccd9f414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"ceb8cb30e7701f771e3116e5ccd9f414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.213135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.213073    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/21665e5cbddb8060242764c579d30712-config\") pod \"kube-apiserver-proxy-ip-10-0-136-40.ec2.internal\" (UID: \"21665e5cbddb8060242764c579d30712\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.213135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.213094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ceb8cb30e7701f771e3116e5ccd9f414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"ceb8cb30e7701f771e3116e5ccd9f414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.213202 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.213143    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ceb8cb30e7701f771e3116e5ccd9f414-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"ceb8cb30e7701f771e3116e5ccd9f414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.213202 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.213150    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ceb8cb30e7701f771e3116e5ccd9f414-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"ceb8cb30e7701f771e3116e5ccd9f414\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.213202 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.213154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/21665e5cbddb8060242764c579d30712-config\") pod \"kube-apiserver-proxy-ip-10-0-136-40.ec2.internal\" (UID: \"21665e5cbddb8060242764c579d30712\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.223351 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.223314    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.324179 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.324085    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.398009 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.397977    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.400314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.400289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:58.424558 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.424516    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.525731 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.525668    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.626830 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.626744    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.721866 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.721838    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 15:10:58.727924 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.727906    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.746471 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.746436    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 04 15:10:58.746646 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.746621    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 15:10:58.746713 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.746629    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 04 15:10:58.828811 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.828765    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.856291 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.856264    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 15:10:58.870165 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.870118    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 15:05:58 +0000 UTC" deadline="2028-01-04 05:36:07.283171819 +0000 UTC"
Aug 04 15:10:58.870165 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.870163    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12422h25m8.413012409s"
Aug 04 15:10:58.928485 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.928389    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 04 15:10:58.928891 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:58.928870    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Aug 04 15:10:58.929547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.929530    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Aug 04 15:10:58.929595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.929565    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Aug 04 15:10:58.929595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.929578    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Aug 04 15:10:58.962856 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:58.962828    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 15:10:59.010755 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.010709    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:59.052209 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.052173    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 15:10:59.053023 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.053007    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Aug 04 15:10:59.082564 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.082532    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 04 15:10:59.216062 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:59.216026    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21665e5cbddb8060242764c579d30712.slice/crio-40efd0e909e5d9c9702375e8f6882f4361d3f442dfeb8198a9810eb85c803035 WatchSource:0}: Error finding container 40efd0e909e5d9c9702375e8f6882f4361d3f442dfeb8198a9810eb85c803035: Status 404 returned error can't find the container with id 40efd0e909e5d9c9702375e8f6882f4361d3f442dfeb8198a9810eb85c803035
Aug 04 15:10:59.216293 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:10:59.216269    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb8cb30e7701f771e3116e5ccd9f414.slice/crio-e043fa1b258b70103f6caac90522e943c2b7c750015a3fb998e3b1d10b2ef0c2 WatchSource:0}: Error finding container e043fa1b258b70103f6caac90522e943c2b7c750015a3fb998e3b1d10b2ef0c2: Status 404 returned error can't find the container with id e043fa1b258b70103f6caac90522e943c2b7c750015a3fb998e3b1d10b2ef0c2
Aug 04 15:10:59.220745 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.220725    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 15:10:59.792947 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.792906    2569 apiserver.go:52] "Watching apiserver"
Aug 04 15:10:59.802389 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.802282    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 04 15:10:59.803026 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.802995    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal","openshift-cluster-node-tuning-operator/tuned-2tbgc","openshift-dns/node-resolver-8xmz4","openshift-image-registry/node-ca-kgs4p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal","openshift-multus/network-metrics-daemon-vj74l","openshift-network-diagnostics/network-check-target-tc5l7","openshift-network-operator/iptables-alerter-pr968","kube-system/global-pull-secret-syncer-bzd8c","kube-system/konnectivity-agent-zcvxt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h","openshift-multus/multus-7k44s","openshift-multus/multus-additional-cni-plugins-sfwpj","openshift-ovn-kubernetes/ovnkube-node-dlwvs"]
Aug 04 15:10:59.805915 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.805817    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.808739 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.808688    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.809876 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.809854    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k844d\""
Aug 04 15:10:59.810077 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.810053    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 04 15:10:59.810205 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.810129    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 04 15:10:59.810277 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.810209    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 04 15:10:59.811979 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.811955    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gwmsq\""
Aug 04 15:10:59.812200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.812173    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.812311 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.812203    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.813148 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.813095    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.814395 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.814370    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.814673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.814646    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.814871 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.814852    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 04 15:10:59.814991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.814968    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l7zmw\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.816420    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.816647    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.816853    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.817074    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.817414    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.818285    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.818484    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.818734    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-894nd\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.818924    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.819267    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.819673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.819508    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 04 15:10:59.820824 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.820802    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 04 15:10:59.821056 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.821036    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:10:59.821145 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.821078    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5q6x7\""
Aug 04 15:10:59.821145 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:59.821122    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:10:59.823368 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.822618    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.824234 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.824234 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-var-lib-kubelet\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.824397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824254    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjv4s\" (UniqueName: \"kubernetes.io/projected/9ebe729a-cf50-43db-90d2-5eb1559585d8-kube-api-access-sjv4s\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.824397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824282    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-cni-netd\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovn-node-metrics-cert\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovnkube-script-lib\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824407    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cni-binary-copy\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-node-log\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-log-socket\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-cni-bin\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824513    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a87dcde-1842-4be7-884a-ab63c3ea6d25-konnectivity-ca\") pod \"konnectivity-agent-zcvxt\" (UID: \"4a87dcde-1842-4be7-884a-ab63c3ea6d25\") " pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-kubernetes\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysctl-conf\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-systemd\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.824726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-lib-modules\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.825379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ebe729a-cf50-43db-90d2-5eb1559585d8-tmp\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.825379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824864    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a87dcde-1842-4be7-884a-ab63c3ea6d25-agent-certs\") pod \"konnectivity-agent-zcvxt\" (UID: \"4a87dcde-1842-4be7-884a-ab63c3ea6d25\") " pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.825379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-run\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.825379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-sys\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.825379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.824980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-systemd\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-var-lib-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825536    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovnkube-config\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-env-overrides\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-system-cni-dir\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825636    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cnibin\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.825672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysconfig\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-kubelet\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-etc-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-ovn\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns4bk\" (UniqueName: \"kubernetes.io/projected/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-kube-api-access-ns4bk\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-os-release\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lr2t\" (UniqueName: \"kubernetes.io/projected/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-kube-api-access-8lr2t\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysctl-d\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825890    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-host\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-run-netns\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825937    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825962    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-modprobe-d\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.825988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-tuned\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.826012    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-systemd-units\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.826036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-slash\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.826248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.826059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.827300 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.826448    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.828146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.828121    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.828365    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.828654    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8445m\""
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.828699    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.829223    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.829290    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.829692    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9kqz9\""
Aug 04 15:10:59.831735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.829750    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 04 15:10:59.834602 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.833799    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.834602 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.834120    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 04 15:10:59.834602 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.834426    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.834813 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.834702    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tlqc2\""
Aug 04 15:10:59.836039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.834986    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 04 15:10:59.836039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.835516    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:10:59.836039 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:59.835582    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:10:59.836039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.835597    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:10:59.836039 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:10:59.835696    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:10:59.838386 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.838160    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:10:59.840675 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.840653    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hwc6n\""
Aug 04 15:10:59.841278 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.840977    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 04 15:10:59.841740 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.841585    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 04 15:10:59.864020 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.863960    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-svhfn"
Aug 04 15:10:59.875175 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.873820    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-svhfn"
Aug 04 15:10:59.911669 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.911642    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 04 15:10:59.926798 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysctl-conf\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.926798 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-systemd\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-lib-modules\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926853    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-var-lib-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a87dcde-1842-4be7-884a-ab63c3ea6d25-agent-certs\") pod \"konnectivity-agent-zcvxt\" (UID: \"4a87dcde-1842-4be7-884a-ab63c3ea6d25\") " pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-run\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926942    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovnkube-config\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns4bk\" (UniqueName: \"kubernetes.io/projected/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-kube-api-access-ns4bk\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.926998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-multus-certs\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.927051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927044    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-system-cni-dir\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-etc-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8fef80a-d507-44dc-ba93-7488792125ce-multus-daemon-config\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-etc-kubernetes\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/319e5610-9a08-4abb-b819-a1d286392a65-dbus\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lr2t\" (UniqueName: \"kubernetes.io/projected/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-kube-api-access-8lr2t\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-host\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-run-netns\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927329    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-system-cni-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927350    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-cnibin\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927368    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-sys-fs\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/319e5610-9a08-4abb-b819-a1d286392a65-kubelet-config\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927407    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-modprobe-d\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-systemd-units\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-slash\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.927554 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-os-release\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-device-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6dmh\" (UniqueName: \"kubernetes.io/projected/147f4ebf-ef58-4c19-a0df-4948c85d78f7-kube-api-access-l6dmh\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927595    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927687    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-var-lib-kubelet\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjv4s\" (UniqueName: \"kubernetes.io/projected/9ebe729a-cf50-43db-90d2-5eb1559585d8-kube-api-access-sjv4s\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovn-node-metrics-cert\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-cni-bin\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-conf-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927864    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovnkube-script-lib\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-hostroot\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927948    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/147f4ebf-ef58-4c19-a0df-4948c85d78f7-hosts-file\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.927981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cni-binary-copy\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8fef80a-d507-44dc-ba93-7488792125ce-cni-binary-copy\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-kubelet\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.928479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928092    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a87dcde-1842-4be7-884a-ab63c3ea6d25-konnectivity-ca\") pod \"konnectivity-agent-zcvxt\" (UID: \"4a87dcde-1842-4be7-884a-ab63c3ea6d25\") " pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-kubernetes\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ebe729a-cf50-43db-90d2-5eb1559585d8-tmp\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-cni-multus\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-registration-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-sys\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-systemd\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928295    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-env-overrides\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928352    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c289d13f-6403-4f24-aaf7-d41b37f7fd98-iptables-alerter-script\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c289d13f-6403-4f24-aaf7-d41b37f7fd98-host-slash\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cnibin\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysconfig\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-kubelet\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-ovn\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.929341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-k8s-cni-cncf-io\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-os-release\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928645    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysctl-d\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928670    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-cni-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a16acf50-04d8-4549-9350-2061249f53c5-host\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxvk\" (UniqueName: \"kubernetes.io/projected/c289d13f-6403-4f24-aaf7-d41b37f7fd98-kube-api-access-flxvk\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-tuned\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928850    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-socket-dir-parent\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-run-netns\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928958    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qqtb\" (UniqueName: \"kubernetes.io/projected/a8fef80a-d507-44dc-ba93-7488792125ce-kube-api-access-8qqtb\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92c47\" (UniqueName: \"kubernetes.io/projected/f8b016ed-b95c-4ffc-897b-478f53ba0203-kube-api-access-92c47\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-modprobe-d\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-socket-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929120    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-systemd-units\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/147f4ebf-ef58-4c19-a0df-4948c85d78f7-tmp-dir\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:10:59.930192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-cni-netd\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysctl-conf\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929297    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-netns\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-systemd\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929394    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wv7t\" (UniqueName: \"kubernetes.io/projected/c5f0f777-a6b8-4866-82c9-4e623dd49137-kube-api-access-4wv7t\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929422    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a16acf50-04d8-4549-9350-2061249f53c5-serviceca\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-node-log\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-log-socket\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929487    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-lib-modules\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929516    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-cni-bin\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929553    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-var-lib-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9f4r\" (UniqueName: \"kubernetes.io/projected/a16acf50-04d8-4549-9350-2061249f53c5-kube-api-access-k9f4r\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929614    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-slash\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.929682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.930180    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.928818    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-host\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.931039 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.930807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-run\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.932009 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.931407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovnkube-config\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.932009 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.931779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-system-cni-dir\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.932009 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.931824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-etc-openvswitch\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.932009 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.931902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-var-lib-kubelet\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.932291 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.932259    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-kubelet\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.932419 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.932307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-ovn\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.932419 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.932391    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-os-release\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.932548 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.932531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysctl-d\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.932740 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.932702    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.933074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovnkube-script-lib\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.933074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.933204 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933118    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-cni-netd\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.933204 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-node-log\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.933351 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-cni-bin\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.933351 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933285    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-log-socket\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.933515 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933378    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-kubernetes\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.933670 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cni-binary-copy\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.933766 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933734    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cnibin\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.934217 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933887    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-env-overrides\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.934217 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-sysconfig\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.934217 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933937    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-run-systemd\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.934217 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.933986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4a87dcde-1842-4be7-884a-ab63c3ea6d25-konnectivity-ca\") pod \"konnectivity-agent-zcvxt\" (UID: \"4a87dcde-1842-4be7-884a-ab63c3ea6d25\") " pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.934217 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.934035    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ebe729a-cf50-43db-90d2-5eb1559585d8-sys\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.934217 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.934169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.934585 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.934349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.934585 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.934571    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4a87dcde-1842-4be7-884a-ab63c3ea6d25-agent-certs\") pod \"konnectivity-agent-zcvxt\" (UID: \"4a87dcde-1842-4be7-884a-ab63c3ea6d25\") " pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:10:59.936801 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.936436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9ebe729a-cf50-43db-90d2-5eb1559585d8-etc-tuned\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.938737 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.938678    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ebe729a-cf50-43db-90d2-5eb1559585d8-tmp\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.938845 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.938786    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-ovn-node-metrics-cert\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.945407 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.945268    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lr2t\" (UniqueName: \"kubernetes.io/projected/3c0d8e72-d822-4540-8a40-9904b0f0cdbe-kube-api-access-8lr2t\") pod \"multus-additional-cni-plugins-sfwpj\" (UID: \"3c0d8e72-d822-4540-8a40-9904b0f0cdbe\") " pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:10:59.945870 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.945645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjv4s\" (UniqueName: \"kubernetes.io/projected/9ebe729a-cf50-43db-90d2-5eb1559585d8-kube-api-access-sjv4s\") pod \"tuned-2tbgc\" (UID: \"9ebe729a-cf50-43db-90d2-5eb1559585d8\") " pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:10:59.946565 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.946539    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns4bk\" (UniqueName: \"kubernetes.io/projected/ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d-kube-api-access-ns4bk\") pod \"ovnkube-node-dlwvs\" (UID: \"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:10:59.964291 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.964184    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"ceb8cb30e7701f771e3116e5ccd9f414","Type":"ContainerStarted","Data":"e043fa1b258b70103f6caac90522e943c2b7c750015a3fb998e3b1d10b2ef0c2"}
Aug 04 15:10:59.966712 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:10:59.966654    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal" event={"ID":"21665e5cbddb8060242764c579d30712","Type":"ContainerStarted","Data":"40efd0e909e5d9c9702375e8f6882f4361d3f442dfeb8198a9810eb85c803035"}
Aug 04 15:11:00.030055 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030013    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:00.030264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-k8s-cni-cncf-io\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-cni-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-k8s-cni-cncf-io\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030264 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.030200    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:00.030264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030205    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-cni-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030227    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a16acf50-04d8-4549-9350-2061249f53c5-host\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.030275    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:00.530257188 +0000 UTC m=+3.036555228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030330    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a16acf50-04d8-4549-9350-2061249f53c5-host\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flxvk\" (UniqueName: \"kubernetes.io/projected/c289d13f-6403-4f24-aaf7-d41b37f7fd98-kube-api-access-flxvk\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-socket-dir-parent\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qqtb\" (UniqueName: \"kubernetes.io/projected/a8fef80a-d507-44dc-ba93-7488792125ce-kube-api-access-8qqtb\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030424    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92c47\" (UniqueName: \"kubernetes.io/projected/f8b016ed-b95c-4ffc-897b-478f53ba0203-kube-api-access-92c47\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030474    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-socket-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030525    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/147f4ebf-ef58-4c19-a0df-4948c85d78f7-tmp-dir\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-netns\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.030593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030605    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wv7t\" (UniqueName: \"kubernetes.io/projected/c5f0f777-a6b8-4866-82c9-4e623dd49137-kube-api-access-4wv7t\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030632    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a16acf50-04d8-4549-9350-2061249f53c5-serviceca\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-socket-dir-parent\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9f4r\" (UniqueName: \"kubernetes.io/projected/a16acf50-04d8-4549-9350-2061249f53c5-kube-api-access-k9f4r\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-multus-certs\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030745    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8fef80a-d507-44dc-ba93-7488792125ce-multus-daemon-config\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-etc-kubernetes\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.030811    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030866    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-netns\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.030886    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:00.530864655 +0000 UTC m=+3.037162704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030927    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/319e5610-9a08-4abb-b819-a1d286392a65-dbus\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-system-cni-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-socket-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.030991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-cnibin\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.031208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031020    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/147f4ebf-ef58-4c19-a0df-4948c85d78f7-tmp-dir\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031040    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-cnibin\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031034    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-sys-fs\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-run-multus-certs\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/319e5610-9a08-4abb-b819-a1d286392a65-kubelet-config\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031151    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/319e5610-9a08-4abb-b819-a1d286392a65-kubelet-config\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031185    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/319e5610-9a08-4abb-b819-a1d286392a65-dbus\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031240    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-system-cni-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031292    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-sys-fs\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031332    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-etc-kubernetes\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-os-release\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-device-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6dmh\" (UniqueName: \"kubernetes.io/projected/147f4ebf-ef58-4c19-a0df-4948c85d78f7-kube-api-access-l6dmh\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-cni-bin\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-conf-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a16acf50-04d8-4549-9350-2061249f53c5-serviceca\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-hostroot\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.032043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031564    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-hostroot\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031566    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/147f4ebf-ef58-4c19-a0df-4948c85d78f7-hosts-file\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031597    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8fef80a-d507-44dc-ba93-7488792125ce-cni-binary-copy\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/147f4ebf-ef58-4c19-a0df-4948c85d78f7-hosts-file\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031624    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-device-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031476    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-os-release\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-kubelet\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-cni-bin\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-cni-multus\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031785    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-kubelet\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031791    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8fef80a-d507-44dc-ba93-7488792125ce-multus-daemon-config\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031796    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-multus-conf-dir\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8fef80a-d507-44dc-ba93-7488792125ce-host-var-lib-cni-multus\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031857    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-registration-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031890    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c289d13f-6403-4f24-aaf7-d41b37f7fd98-iptables-alerter-script\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031930    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5f0f777-a6b8-4866-82c9-4e623dd49137-registration-dir\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c289d13f-6403-4f24-aaf7-d41b37f7fd98-host-slash\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.033006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.031978    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c289d13f-6403-4f24-aaf7-d41b37f7fd98-host-slash\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.033927 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.032189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8fef80a-d507-44dc-ba93-7488792125ce-cni-binary-copy\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.033927 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.033057    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c289d13f-6403-4f24-aaf7-d41b37f7fd98-iptables-alerter-script\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.046283 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.046194    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:00.046283 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.046228    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:00.046283 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.046247    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:00.046575 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.046364    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:00.546339673 +0000 UTC m=+3.052637712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:00.049233 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.049195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wv7t\" (UniqueName: \"kubernetes.io/projected/c5f0f777-a6b8-4866-82c9-4e623dd49137-kube-api-access-4wv7t\") pod \"aws-ebs-csi-driver-node-fxp5h\" (UID: \"c5f0f777-a6b8-4866-82c9-4e623dd49137\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.050543 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.050501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92c47\" (UniqueName: \"kubernetes.io/projected/f8b016ed-b95c-4ffc-897b-478f53ba0203-kube-api-access-92c47\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:00.051638 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.051612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qqtb\" (UniqueName: \"kubernetes.io/projected/a8fef80a-d507-44dc-ba93-7488792125ce-kube-api-access-8qqtb\") pod \"multus-7k44s\" (UID: \"a8fef80a-d507-44dc-ba93-7488792125ce\") " pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.052872 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.052848    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxvk\" (UniqueName: \"kubernetes.io/projected/c289d13f-6403-4f24-aaf7-d41b37f7fd98-kube-api-access-flxvk\") pod \"iptables-alerter-pr968\" (UID: \"c289d13f-6403-4f24-aaf7-d41b37f7fd98\") " pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.059616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.059560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6dmh\" (UniqueName: \"kubernetes.io/projected/147f4ebf-ef58-4c19-a0df-4948c85d78f7-kube-api-access-l6dmh\") pod \"node-resolver-8xmz4\" (UID: \"147f4ebf-ef58-4c19-a0df-4948c85d78f7\") " pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.060062 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.060044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9f4r\" (UniqueName: \"kubernetes.io/projected/a16acf50-04d8-4549-9350-2061249f53c5-kube-api-access-k9f4r\") pod \"node-ca-kgs4p\" (UID: \"a16acf50-04d8-4549-9350-2061249f53c5\") " pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.125532 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.125492    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:11:00.140136 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.140107    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2tbgc"
Aug 04 15:11:00.152938 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.152909    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:11:00.153500 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.153459    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebe729a_cf50_43db_90d2_5eb1559585d8.slice/crio-9a2259f14820f162b3bb994db3c33949367177974fb62b6fc09c36a91b8e238f WatchSource:0}: Error finding container 9a2259f14820f162b3bb994db3c33949367177974fb62b6fc09c36a91b8e238f: Status 404 returned error can't find the container with id 9a2259f14820f162b3bb994db3c33949367177974fb62b6fc09c36a91b8e238f
Aug 04 15:11:00.161388 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.161360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sfwpj"
Aug 04 15:11:00.165435 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.165404    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4d686c_b6e6_4cd5_ba34_7d5e2c65127d.slice/crio-b808b0f8f5887ccebc3e385d40d51abf291a28017816b42d5b3dceba578fc037 WatchSource:0}: Error finding container b808b0f8f5887ccebc3e385d40d51abf291a28017816b42d5b3dceba578fc037: Status 404 returned error can't find the container with id b808b0f8f5887ccebc3e385d40d51abf291a28017816b42d5b3dceba578fc037
Aug 04 15:11:00.169656 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.169628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7k44s"
Aug 04 15:11:00.174168 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.174139    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0d8e72_d822_4540_8a40_9904b0f0cdbe.slice/crio-59ad740bb8614eae85d4076c5a31cdb13090cbf94f419f7a7d6f073efeafcc12 WatchSource:0}: Error finding container 59ad740bb8614eae85d4076c5a31cdb13090cbf94f419f7a7d6f073efeafcc12: Status 404 returned error can't find the container with id 59ad740bb8614eae85d4076c5a31cdb13090cbf94f419f7a7d6f073efeafcc12
Aug 04 15:11:00.180214 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.180186    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h"
Aug 04 15:11:00.181531 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.181495    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fef80a_d507_44dc_ba93_7488792125ce.slice/crio-a6457d67e6e21e10e9c98f09e38068fe7084d0bea0141390ecc79e4a336a4ce5 WatchSource:0}: Error finding container a6457d67e6e21e10e9c98f09e38068fe7084d0bea0141390ecc79e4a336a4ce5: Status 404 returned error can't find the container with id a6457d67e6e21e10e9c98f09e38068fe7084d0bea0141390ecc79e4a336a4ce5
Aug 04 15:11:00.190784 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.190494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pr968"
Aug 04 15:11:00.192813 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.192772    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f0f777_a6b8_4866_82c9_4e623dd49137.slice/crio-90c381a7b911f2aec9fb52eb4793fd51bc1b2d31c626cfdbc7b50bc0592dc9be WatchSource:0}: Error finding container 90c381a7b911f2aec9fb52eb4793fd51bc1b2d31c626cfdbc7b50bc0592dc9be: Status 404 returned error can't find the container with id 90c381a7b911f2aec9fb52eb4793fd51bc1b2d31c626cfdbc7b50bc0592dc9be
Aug 04 15:11:00.201261 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.201080    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kgs4p"
Aug 04 15:11:00.206557 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.206519    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc289d13f_6403_4f24_aaf7_d41b37f7fd98.slice/crio-e069e76414f97fce1b7cf2cee5e2e301f146158d240d759cedd47dfbe0cfc73a WatchSource:0}: Error finding container e069e76414f97fce1b7cf2cee5e2e301f146158d240d759cedd47dfbe0cfc73a: Status 404 returned error can't find the container with id e069e76414f97fce1b7cf2cee5e2e301f146158d240d759cedd47dfbe0cfc73a
Aug 04 15:11:00.208277 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.208063    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xmz4"
Aug 04 15:11:00.219485 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.219458    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 15:11:00.220528 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:00.220499    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147f4ebf_ef58_4c19_a0df_4948c85d78f7.slice/crio-5e579f1e9cb2fdf529ec78508a4f918eaaadb74603b540001efb887fcb43a470 WatchSource:0}: Error finding container 5e579f1e9cb2fdf529ec78508a4f918eaaadb74603b540001efb887fcb43a470: Status 404 returned error can't find the container with id 5e579f1e9cb2fdf529ec78508a4f918eaaadb74603b540001efb887fcb43a470
Aug 04 15:11:00.536147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.536059    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:00.536147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.536100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:00.536368 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.536209    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:00.536368 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.536279    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:01.536265301 +0000 UTC m=+4.042563358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:00.536368 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.536210    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:00.536564 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.536384    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:01.536362694 +0000 UTC m=+4.042660738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:00.637463 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.637421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:00.637647 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.637556    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:00.637647 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.637569    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:00.637647 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.637578    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:00.637647 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.637635    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:01.637620318 +0000 UTC m=+4.143918354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:00.878255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.878112    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 15:06:00 +0000 UTC" deadline="2028-02-14 15:05:01.571870827 +0000 UTC"
Aug 04 15:11:00.878255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.878198    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13415h54m0.693677463s"
Aug 04 15:11:00.959324 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.959283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:00.959521 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:00.959440    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:00.970367 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.970322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xmz4" event={"ID":"147f4ebf-ef58-4c19-a0df-4948c85d78f7","Type":"ContainerStarted","Data":"5e579f1e9cb2fdf529ec78508a4f918eaaadb74603b540001efb887fcb43a470"}
Aug 04 15:11:00.971898 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.971863    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kgs4p" event={"ID":"a16acf50-04d8-4549-9350-2061249f53c5","Type":"ContainerStarted","Data":"51781e1c1afce7974a391082ce2886a65f1d127859b287273fcaefa1dc5c8fea"}
Aug 04 15:11:00.973391 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.973359    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pr968" event={"ID":"c289d13f-6403-4f24-aaf7-d41b37f7fd98","Type":"ContainerStarted","Data":"e069e76414f97fce1b7cf2cee5e2e301f146158d240d759cedd47dfbe0cfc73a"}
Aug 04 15:11:00.975694 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.975645    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h" event={"ID":"c5f0f777-a6b8-4866-82c9-4e623dd49137","Type":"ContainerStarted","Data":"90c381a7b911f2aec9fb52eb4793fd51bc1b2d31c626cfdbc7b50bc0592dc9be"}
Aug 04 15:11:00.977298 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.977268    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerStarted","Data":"59ad740bb8614eae85d4076c5a31cdb13090cbf94f419f7a7d6f073efeafcc12"}
Aug 04 15:11:00.978666 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.978642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2tbgc" event={"ID":"9ebe729a-cf50-43db-90d2-5eb1559585d8","Type":"ContainerStarted","Data":"9a2259f14820f162b3bb994db3c33949367177974fb62b6fc09c36a91b8e238f"}
Aug 04 15:11:00.980032 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.980000    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7k44s" event={"ID":"a8fef80a-d507-44dc-ba93-7488792125ce","Type":"ContainerStarted","Data":"a6457d67e6e21e10e9c98f09e38068fe7084d0bea0141390ecc79e4a336a4ce5"}
Aug 04 15:11:00.981363 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.981341    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"b808b0f8f5887ccebc3e385d40d51abf291a28017816b42d5b3dceba578fc037"}
Aug 04 15:11:00.982672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:00.982623    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zcvxt" event={"ID":"4a87dcde-1842-4be7-884a-ab63c3ea6d25","Type":"ContainerStarted","Data":"262df6e3100c3aa439dd3ac8624fe22dd7d9cf11d5d8d81db94577defbf7c423"}
Aug 04 15:11:01.005994 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.005933    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 04 15:11:01.546682 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.546356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:01.546915 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.546730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:01.546915 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.546906    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:01.547036 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.547000    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:03.546973067 +0000 UTC m=+6.053271132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:01.547388 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.547362    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:01.547503 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.547440    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:03.547419695 +0000 UTC m=+6.053717740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:01.650474 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.647479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:01.650474 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.647716    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:01.650474 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.647745    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:01.650474 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.647761    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:01.650474 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.647836    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:03.647815287 +0000 UTC m=+6.154113325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:01.878958 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.878819    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 15:06:00 +0000 UTC" deadline="2028-03-21 02:34:57.884129853 +0000 UTC"
Aug 04 15:11:01.878958 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.878865    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14267h23m56.005269816s"
Aug 04 15:11:01.960181 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.960052    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:01.960380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:01.960255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:01.960380 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.960249    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:01.960380 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:01.960368    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:02.004944 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:02.004217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal" event={"ID":"21665e5cbddb8060242764c579d30712","Type":"ContainerStarted","Data":"259fd31a21990343947729709516f99ffb762c1b60c4d1ff41217866aaa4d3ba"}
Aug 04 15:11:02.958892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:02.958822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:02.959502 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:02.958985    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:03.016621 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.016582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"ceb8cb30e7701f771e3116e5ccd9f414","Type":"ContainerStarted","Data":"e7f1f659ade1e7ee3d1b8a4a6ec6dede15ad4afb8760a721e3531ebbd31de7a6"}
Aug 04 15:11:03.054384 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.054309    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal" podStartSLOduration=4.054284301 podStartE2EDuration="4.054284301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:02.032033335 +0000 UTC m=+4.538331396" watchObservedRunningTime="2026-08-04 15:11:03.054284301 +0000 UTC m=+5.560582362"
Aug 04 15:11:03.571102 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.571056    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:03.571311 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.571134    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:03.571311 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.571290    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:03.571422 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.571363    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:07.571338546 +0000 UTC m=+10.077636584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:03.571900 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.571877    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:03.571983 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.571951    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:07.571933001 +0000 UTC m=+10.078231053 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:03.673155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.672199    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:03.673155 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.672466    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:03.673155 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.672493    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:03.673155 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.672508    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:03.673155 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.672572    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:07.672551157 +0000 UTC m=+10.178849196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:03.961285 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.960792    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:03.961285 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.960936    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:03.961285 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:03.961055    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:03.961285 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:03.961152    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:04.959911 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:04.959864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:04.960170 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:04.960044    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:05.958958 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:05.958915    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:05.959564 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:05.959054    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:05.961224 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:05.961191    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:05.961404 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:05.961351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:06.958892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:06.958854    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:06.959098 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:06.959037    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:07.029088 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.028975    2569 generic.go:356] "Generic (PLEG): container finished" podID="ceb8cb30e7701f771e3116e5ccd9f414" containerID="e7f1f659ade1e7ee3d1b8a4a6ec6dede15ad4afb8760a721e3531ebbd31de7a6" exitCode=0
Aug 04 15:11:07.029088 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.029040    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"ceb8cb30e7701f771e3116e5ccd9f414","Type":"ContainerDied","Data":"e7f1f659ade1e7ee3d1b8a4a6ec6dede15ad4afb8760a721e3531ebbd31de7a6"}
Aug 04 15:11:07.607761 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.607719    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:07.607983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.607842    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:07.607983 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.607930    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:07.607983 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.607945    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:07.608174 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.607999    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:15.607980074 +0000 UTC m=+18.114278115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:07.608174 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.608017    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:15.608008467 +0000 UTC m=+18.114306509 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:07.708952 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.708911    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:07.709168 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.709131    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:07.709168 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.709161    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:07.709283 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.709176    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:07.709283 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.709248    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:15.709226008 +0000 UTC m=+18.215524063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:07.965396 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.962353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:07.965396 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.962546    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:07.965396 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:07.962658    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:07.965396 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:07.963334    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:08.958827 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:08.958785    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:08.958996 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:08.958934    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:09.960031 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:09.958820    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:09.960031 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:09.958836    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:09.960031 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:09.958961    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:09.960031 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:09.959117    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:10.959150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:10.959121    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:10.959324 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:10.959285    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:11.962410 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:11.962381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:11.962872 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:11.962494    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:11.962872 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:11.962575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:11.962872 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:11.962706    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:12.959715 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:12.959675    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:12.959919 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:12.959821    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:13.959476 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:13.959435    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:13.959930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:13.959483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:13.959930 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:13.959577    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:13.959930 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:13.959720    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:14.959110 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:14.959070    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:14.959345 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:14.959218    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:15.667157 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:15.667107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:15.667157 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:15.667165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:15.667676 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.667267    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:15.667676 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.667289    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:15.667676 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.667342    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:31.66732085 +0000 UTC m=+34.173618896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:15.667676 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.667362    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:31.667351546 +0000 UTC m=+34.173649586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:15.768479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:15.768430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:15.768694 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.768607    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:15.768694 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.768629    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:15.768694 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.768656    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:15.768917 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.768719    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:11:31.768701443 +0000 UTC m=+34.274999500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:15.959908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:15.959823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:15.960076 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.959956    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:15.960076 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:15.959979    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:15.960183 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:15.960093    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:16.958909 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:16.958873    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:16.959353 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:16.958998    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:17.960014 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:17.959977    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:17.960469 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:17.960087    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:17.960469 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:17.960134    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:17.960469 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:17.960248    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:18.959748 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:18.959713    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:18.959929 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:18.959870    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:19.959146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:19.959105    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:19.959659 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:19.959212    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:19.959659 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:19.959600    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:19.959751 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:19.959693    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:20.055661 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.055291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2tbgc" event={"ID":"9ebe729a-cf50-43db-90d2-5eb1559585d8","Type":"ContainerStarted","Data":"735ad8aa1acd2a06e4bc10ca3cc34f36f2b006e7d75fe3e30683b91615380ff2"}
Aug 04 15:11:20.057968 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.057945    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"ceb8cb30e7701f771e3116e5ccd9f414","Type":"ContainerStarted","Data":"7200a7e11c536d1d45c226365c9e2a099d841c7434e371e85c5f1bdc6b0c9b1a"}
Aug 04 15:11:20.061625 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.060059    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zcvxt" event={"ID":"4a87dcde-1842-4be7-884a-ab63c3ea6d25","Type":"ContainerStarted","Data":"35f124997cbd93b7c5466ffd931eb58f99f991bc9c8072beb11829cb568f7469"}
Aug 04 15:11:20.063801 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.063778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h" event={"ID":"c5f0f777-a6b8-4866-82c9-4e623dd49137","Type":"ContainerStarted","Data":"4f07768d37b20488654e54a2cb207cf8701833846d25c3cec8e19efb58ca0aa5"}
Aug 04 15:11:20.080161 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.080102    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2tbgc" podStartSLOduration=2.524698252 podStartE2EDuration="22.080081879s" totalImagesPullingTime="19.555383627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:20.080004596 +0000 UTC m=+22.586302652" watchObservedRunningTime="2026-08-04 15:11:20.080081879 +0000 UTC m=+22.586379935"
Aug 04 15:11:20.099974 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.099733    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" podStartSLOduration=21.099718446 podStartE2EDuration="21.099718446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:20.099446538 +0000 UTC m=+22.605744595" watchObservedRunningTime="2026-08-04 15:11:20.099718446 +0000 UTC m=+22.606016501"
Aug 04 15:11:20.959845 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:20.959566    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:20.960507 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:20.959901    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:21.067725 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.067688    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7k44s" event={"ID":"a8fef80a-d507-44dc-ba93-7488792125ce","Type":"ContainerStarted","Data":"7a82fe7bdc5c00e21f53440d64569c14a0296dfa8da4877f11d38423e06523c5"}
Aug 04 15:11:21.070650 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.070609    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"160929e53d6f2e29b38a430efc4b65b13ce9740742f579f7e6f337f4ab09bc29"}
Aug 04 15:11:21.070802 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.070706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"52f1aed06746df5e6823d75c439bf1db3dfa62994c69120b48b32381b8f44e69"}
Aug 04 15:11:21.070802 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.070757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"912b07e82c657b6a1d3408868a4872f3019a0f8bc2113acd563f848e5f32466f"}
Aug 04 15:11:21.070920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.070801    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"a7b4a1e2a0453f94193418bd7fd87d56da83b9a4ae80fb2f44b84a953994f5ce"}
Aug 04 15:11:21.070920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.070847    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"8c73ebc9adfec0b9dc253303e2b427f1caf6cae6409f49ad7d72fb4a94559fee"}
Aug 04 15:11:21.070920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.070894    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"755291b47436aca9b94c2b2273d1c02b72be83a9a62c2979587e6e7b1119bb2c"}
Aug 04 15:11:21.071926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.071904    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xmz4" event={"ID":"147f4ebf-ef58-4c19-a0df-4948c85d78f7","Type":"ContainerStarted","Data":"de8296d0a2cb33ac93a25dd716b3894bd5c2045600b110904c3974a46e49c814"}
Aug 04 15:11:21.073314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.073291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kgs4p" event={"ID":"a16acf50-04d8-4549-9350-2061249f53c5","Type":"ContainerStarted","Data":"1b34840c408a100048e8531fa1cf43e198572784c5d46f78d1e03637b0752be8"}
Aug 04 15:11:21.074796 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.074774    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0d8e72-d822-4540-8a40-9904b0f0cdbe" containerID="584341dfe022f4285b3dfe6cbb7c4ed907106fab5a16f74893694a3b7b203b85" exitCode=0
Aug 04 15:11:21.074899 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.074852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerDied","Data":"584341dfe022f4285b3dfe6cbb7c4ed907106fab5a16f74893694a3b7b203b85"}
Aug 04 15:11:21.092388 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.092341    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zcvxt" podStartSLOduration=12.009981303 podStartE2EDuration="23.092326248s" totalImagesPullingTime="11.082344945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:20.118764417 +0000 UTC m=+22.625062475" watchObservedRunningTime="2026-08-04 15:11:21.092326248 +0000 UTC m=+23.598624304"
Aug 04 15:11:21.092582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.092475    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7k44s" podStartSLOduration=3.560498053 podStartE2EDuration="23.092471679s" totalImagesPullingTime="19.531973626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:21.092099728 +0000 UTC m=+23.598397784" watchObservedRunningTime="2026-08-04 15:11:21.092471679 +0000 UTC m=+23.598769735"
Aug 04 15:11:21.149871 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.149821    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kgs4p" podStartSLOduration=2.685118972 podStartE2EDuration="22.149805388s" totalImagesPullingTime="19.464686416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:21.149170811 +0000 UTC m=+23.655468892" watchObservedRunningTime="2026-08-04 15:11:21.149805388 +0000 UTC m=+23.656103444"
Aug 04 15:11:21.168711 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.168658    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8xmz4" podStartSLOduration=2.68126788 podStartE2EDuration="22.16862315s" totalImagesPullingTime="19.48735527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:21.168253436 +0000 UTC m=+23.674551492" watchObservedRunningTime="2026-08-04 15:11:21.16862315 +0000 UTC m=+23.674921205"
Aug 04 15:11:21.388274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.388243    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 04 15:11:21.894873 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.894742    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T15:11:21.388266426Z","UUID":"b935ddef-67e8-46a3-a167-edba745c1a6b","Handler":null,"Name":"","Endpoint":""}
Aug 04 15:11:21.896416 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.896393    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 04 15:11:21.896576 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.896438    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 04 15:11:21.959334 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.959297    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:21.959534 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:21.959297    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:21.959534 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:21.959447    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:21.959534 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:21.959467    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:22.079405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:22.079363    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h" event={"ID":"c5f0f777-a6b8-4866-82c9-4e623dd49137","Type":"ContainerStarted","Data":"d36a4a03d864c0c8283506b110721acb070067c09644b17a1cdbd805a7c679a2"}
Aug 04 15:11:22.081136 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:22.081050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pr968" event={"ID":"c289d13f-6403-4f24-aaf7-d41b37f7fd98","Type":"ContainerStarted","Data":"0707c67ecc71c188d211709c12061dc97649623a993146617d7f4b3fa40c6649"}
Aug 04 15:11:22.105926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:22.105864    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pr968" podStartSLOduration=5.039997137 podStartE2EDuration="24.10584878s" totalImagesPullingTime="19.065851643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:22.10549154 +0000 UTC m=+24.611789597" watchObservedRunningTime="2026-08-04 15:11:22.10584878 +0000 UTC m=+24.612146836"
Aug 04 15:11:22.669750 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:22.669697    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:11:22.670475 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:22.670446    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:11:22.958956 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:22.958872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:22.959098 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:22.959000    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:23.088856 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:23.088787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"719f7d5b52748124f6f6f43af23f3fa0f7fdf736a001c3e2d16d2f215278cbeb"}
Aug 04 15:11:23.958892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:23.958859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:23.959099 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:23.958859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:23.959099 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:23.958991    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:23.959211 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:23.959089    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:24.093094 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:24.093052    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h" event={"ID":"c5f0f777-a6b8-4866-82c9-4e623dd49137","Type":"ContainerStarted","Data":"e5cfa6a94c49cfe1a4e12d6db58f34d60e3977d3f521abf73570ff0d6a4771a0"}
Aug 04 15:11:24.117742 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:24.117688    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp5h" podStartSLOduration=12.055372652 podStartE2EDuration="26.117670964s" totalImagesPullingTime="14.062298312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:24.117398858 +0000 UTC m=+26.623696913" watchObservedRunningTime="2026-08-04 15:11:24.117670964 +0000 UTC m=+26.623969021"
Aug 04 15:11:24.959298 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:24.959260    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:24.959494 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:24.959391    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:25.959247 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:25.959063    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:25.959852 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:25.959090    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:25.959852 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:25.959350    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:25.959852 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:25.959389    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:26.100228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.100195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" event={"ID":"ea4d686c-b6e6-4cd5-ba34-7d5e2c65127d","Type":"ContainerStarted","Data":"b6a6d9186f11ed38b8fcb3b5815409e61ba8a2226152ab6cf42a1bb8fffad89f"}
Aug 04 15:11:26.100468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.100444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:11:26.100567 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.100512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:11:26.100567 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.100561    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:11:26.116529 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.116502    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:11:26.116705 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.116614    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:11:26.142716 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.142622    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs" podStartSLOduration=8.344576433 podStartE2EDuration="28.142600308s" totalImagesPullingTime="19.798023875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:26.132510349 +0000 UTC m=+28.638808415" watchObservedRunningTime="2026-08-04 15:11:26.142600308 +0000 UTC m=+28.648898375"
Aug 04 15:11:26.691615 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.691584    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xmz4_147f4ebf-ef58-4c19-a0df-4948c85d78f7/dns-node-resolver/0.log"
Aug 04 15:11:26.959442 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:26.959361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:26.959941 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:26.959481    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:27.105828 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:27.105786    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0d8e72-d822-4540-8a40-9904b0f0cdbe" containerID="9b7f4542be2ab570a8d30f7a272e59f4f878a0da63687167fe2e13435038fbd9" exitCode=0
Aug 04 15:11:27.107427 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:27.107393    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerDied","Data":"9b7f4542be2ab570a8d30f7a272e59f4f878a0da63687167fe2e13435038fbd9"}
Aug 04 15:11:27.471246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:27.471220    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kgs4p_a16acf50-04d8-4549-9350-2061249f53c5/node-ca/0.log"
Aug 04 15:11:27.960156 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:27.959959    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:27.960657 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:27.960031    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:27.960657 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:27.960242    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:27.960657 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:27.960302    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:28.124192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.124150    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tc5l7"]
Aug 04 15:11:28.124477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.124460    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bzd8c"]
Aug 04 15:11:28.124650 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.124606    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:28.124771 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.124760    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:28.124947 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.124919    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vj74l"]
Aug 04 15:11:28.125072 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.125042    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:28.125136 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:28.124918    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:28.125136 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:28.124758    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:28.125250 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:28.125132    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:28.404708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.404626    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:11:28.404890 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.404827    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Aug 04 15:11:28.405610 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:28.405358    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zcvxt"
Aug 04 15:11:29.112336 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:29.112245    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0d8e72-d822-4540-8a40-9904b0f0cdbe" containerID="14c51acc33eb65bf9b535aad0c9bed1b33ef78d42d5b495de9fbd71ff763bb6a" exitCode=0
Aug 04 15:11:29.112837 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:29.112329    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerDied","Data":"14c51acc33eb65bf9b535aad0c9bed1b33ef78d42d5b495de9fbd71ff763bb6a"}
Aug 04 15:11:29.959584 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:29.959543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:29.959770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:29.959543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:29.959770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:29.959694    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:29.959770 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:29.959688    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:29.959933 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:29.959805    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:29.959933 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:29.959890    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:30.117191 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:30.117113    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0d8e72-d822-4540-8a40-9904b0f0cdbe" containerID="2f8fea3a72e14d90dc57402ee4ce3a7531629af9c229585435ea114f85e3ff24" exitCode=0
Aug 04 15:11:30.117191 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:30.117169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerDied","Data":"2f8fea3a72e14d90dc57402ee4ce3a7531629af9c229585435ea114f85e3ff24"}
Aug 04 15:11:31.688721 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:31.688681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:31.688721 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:31.688727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:31.689393 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.688848    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:31.689393 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.688924    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs podName:f8b016ed-b95c-4ffc-897b-478f53ba0203 nodeName:}" failed. No retries permitted until 2026-08-04 15:12:03.688906451 +0000 UTC m=+66.195204484 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs") pod "network-metrics-daemon-vj74l" (UID: "f8b016ed-b95c-4ffc-897b-478f53ba0203") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 04 15:11:31.689393 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.688855    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:31.689393 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.689020    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret podName:319e5610-9a08-4abb-b819-a1d286392a65 nodeName:}" failed. No retries permitted until 2026-08-04 15:12:03.689001456 +0000 UTC m=+66.195299493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret") pod "global-pull-secret-syncer-bzd8c" (UID: "319e5610-9a08-4abb-b819-a1d286392a65") : object "kube-system"/"original-pull-secret" not registered
Aug 04 15:11:31.789521 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:31.789456    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:31.789713 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.789662    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 04 15:11:31.789713 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.789690    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 04 15:11:31.789713 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.789704    2569 projected.go:196] Error preparing data for projected volume kube-api-access-2jxh9 for pod openshift-network-diagnostics/network-check-target-tc5l7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:31.789870 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.789772    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9 podName:0a5095ac-244f-48c3-90f8-de9a5a127ab8 nodeName:}" failed. No retries permitted until 2026-08-04 15:12:03.789752094 +0000 UTC m=+66.296050131 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2jxh9" (UniqueName: "kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9") pod "network-check-target-tc5l7" (UID: "0a5095ac-244f-48c3-90f8-de9a5a127ab8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 04 15:11:31.959454 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:31.959365    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:31.959454 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:31.959409    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:31.959683 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:31.959480    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:31.959683 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.959595    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:31.960017 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.959848    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:31.960017 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:31.959972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:33.959432 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:33.959394    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:33.960076 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:33.959449    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:33.960076 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:33.959395    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:33.960076 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:33.959535    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:33.960076 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:33.959676    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:33.960076 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:33.959769    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:35.132720 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:35.132680    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0d8e72-d822-4540-8a40-9904b0f0cdbe" containerID="f8a32cfee481b027c95425f1cf4223d54cb87a531d7bc0aa64fc3540d1f7c213" exitCode=0
Aug 04 15:11:35.133203 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:35.132744    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerDied","Data":"f8a32cfee481b027c95425f1cf4223d54cb87a531d7bc0aa64fc3540d1f7c213"}
Aug 04 15:11:35.959791 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:35.959751    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:35.959969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:35.959758    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:35.959969 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:35.959899    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:35.959969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:35.959758    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:35.959969 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:35.959953    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:35.960107 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:35.960002    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:36.137875 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:36.137833    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c0d8e72-d822-4540-8a40-9904b0f0cdbe" containerID="7f34698deafb015d7135cfa99f99937027f5e5a528815f27b8126e2055b80699" exitCode=0
Aug 04 15:11:36.138303 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:36.137890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerDied","Data":"7f34698deafb015d7135cfa99f99937027f5e5a528815f27b8126e2055b80699"}
Aug 04 15:11:37.143689 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:37.143461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" event={"ID":"3c0d8e72-d822-4540-8a40-9904b0f0cdbe","Type":"ContainerStarted","Data":"f469b0efda0e8a69019b7f110f6cdf290f75b1cb5ee0d74a5630817cea5acfd4"}
Aug 04 15:11:37.173959 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:37.173905    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sfwpj" podStartSLOduration=8.040518497 podStartE2EDuration="39.173887735s" totalImagesPullingTime="31.133369238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:37.172797814 +0000 UTC m=+39.679095869" watchObservedRunningTime="2026-08-04 15:11:37.173887735 +0000 UTC m=+39.680185791"
Aug 04 15:11:37.960848 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:37.960756    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:37.960986 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:37.960856    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:37.960986 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:37.960898    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:37.960986 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:37.960892    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:37.960986 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:37.960976    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:37.961120 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:37.961088    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:39.959539 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:39.959501    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:39.959539 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:39.959543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:39.960088 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:39.959501    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:39.960088 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:39.959624    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:39.960088 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:39.959772    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:39.960088 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:39.959847    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:41.959259 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:41.959222    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:41.959669 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:41.959222    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:41.959669 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:41.959336    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:41.959669 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:41.959220    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:41.959669 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:41.959406    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:41.959669 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:41.959496    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:43.959528 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:43.959491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:43.959996 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:43.959493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:43.959996 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:43.959599    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:43.959996 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:43.959623    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:43.959996 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:43.959732    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:43.959996 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:43.959797    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:45.959307 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:45.959269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:45.959307 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:45.959292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:45.959875 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:45.959269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:45.959875 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:45.959439    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:45.959875 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:45.959382    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:45.959875 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:45.959504    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:47.960138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:47.960097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:47.960548 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:47.960192    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:47.960548 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:47.960236    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:47.960548 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:47.960271    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:47.960548 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:47.960310    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:47.960548 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:47.960390    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:49.962461 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:49.962421    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:49.962958 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:49.962528    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:49.962958 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:49.962559    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:49.962958 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:49.962545    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tc5l7" podUID="0a5095ac-244f-48c3-90f8-de9a5a127ab8"
Aug 04 15:11:49.962958 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:49.962657    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vj74l" podUID="f8b016ed-b95c-4ffc-897b-478f53ba0203"
Aug 04 15:11:49.962958 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:11:49.962728    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzd8c" podUID="319e5610-9a08-4abb-b819-a1d286392a65"
Aug 04 15:11:50.808804 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.808726    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeReady"
Aug 04 15:11:50.809011 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.808847    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 04 15:11:50.872675 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.872625    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78c985b499-qf2qg"]
Aug 04 15:11:50.896867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.896829    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cncqn"]
Aug 04 15:11:50.897048 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.896969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.899610 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.899589    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Aug 04 15:11:50.899841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.899826    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Aug 04 15:11:50.900296 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.900210    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Aug 04 15:11:50.901335 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.901318    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cqks7\""
Aug 04 15:11:50.905385 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.905360    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Aug 04 15:11:50.916422 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916391    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f8658b5-ba77-48f1-bb28-79360fe4e918-ca-trust-extracted\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.916655 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f8658b5-ba77-48f1-bb28-79360fe4e918-registry-certificates\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.916821 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f8658b5-ba77-48f1-bb28-79360fe4e918-trusted-ca\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.916892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f8658b5-ba77-48f1-bb28-79360fe4e918-image-registry-private-configuration\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.916892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-registry-tls\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.917004 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f8658b5-ba77-48f1-bb28-79360fe4e918-installation-pull-secrets\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.917004 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916926    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxrd2\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-kube-api-access-bxrd2\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.917004 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.916998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-bound-sa-token\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:50.917156 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.917042    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8h4jc"]
Aug 04 15:11:50.917267 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.917247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:50.920916 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.920887    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kdxng\""
Aug 04 15:11:50.921575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.921549    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 04 15:11:50.921821 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.921807    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 04 15:11:50.921882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.921830    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 04 15:11:50.934869 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.934834    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78c985b499-qf2qg"]
Aug 04 15:11:50.934869 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.934868    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cncqn"]
Aug 04 15:11:50.935087 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.934884    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8h4jc"]
Aug 04 15:11:50.935087 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.934973    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:50.938416 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.938392    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 04 15:11:50.938553 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.938434    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 04 15:11:50.939525 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.939505    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 04 15:11:50.939652 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.939523    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ssq9w\""
Aug 04 15:11:50.939765 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.939748    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 04 15:11:50.985687 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:50.985655    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5k478"]
Aug 04 15:11:51.010916 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.010885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5k478"]
Aug 04 15:11:51.011073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.011005    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.014464 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.014436    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 04 15:11:51.014729 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.014704    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 04 15:11:51.014847 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.014765    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xckv6\""
Aug 04 15:11:51.017408 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f8658b5-ba77-48f1-bb28-79360fe4e918-ca-trust-extracted\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f8658b5-ba77-48f1-bb28-79360fe4e918-registry-certificates\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f8658b5-ba77-48f1-bb28-79360fe4e918-trusted-ca\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215b5c0-7b9b-476e-8f79-e46eac80dbca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.017662 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45fa4203-4fe3-4862-a527-401fb126b252-config-volume\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.017662 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59zz5\" (UniqueName: \"kubernetes.io/projected/0f24288a-aa4e-4462-ae64-8cbdbd8c5c42-kube-api-access-59zz5\") pod \"ingress-canary-cncqn\" (UID: \"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42\") " pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.017774 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f8658b5-ba77-48f1-bb28-79360fe4e918-image-registry-private-configuration\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017774 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-registry-tls\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017774 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f8658b5-ba77-48f1-bb28-79360fe4e918-installation-pull-secrets\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017922 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxrd2\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-kube-api-access-bxrd2\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017922 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f24288a-aa4e-4462-ae64-8cbdbd8c5c42-cert\") pod \"ingress-canary-cncqn\" (UID: \"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42\") " pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.017922 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-bound-sa-token\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.017922 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa4203-4fe3-4862-a527-401fb126b252-metrics-tls\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4xql\" (UniqueName: \"kubernetes.io/projected/7215b5c0-7b9b-476e-8f79-e46eac80dbca-kube-api-access-r4xql\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7215b5c0-7b9b-476e-8f79-e46eac80dbca-data-volume\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6f8658b5-ba77-48f1-bb28-79360fe4e918-ca-trust-extracted\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.017983    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpg58\" (UniqueName: \"kubernetes.io/projected/45fa4203-4fe3-4862-a527-401fb126b252-kube-api-access-wpg58\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.018016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7215b5c0-7b9b-476e-8f79-e46eac80dbca-crio-socket\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.018046    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215b5c0-7b9b-476e-8f79-e46eac80dbca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.018119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.018068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45fa4203-4fe3-4862-a527-401fb126b252-tmp-dir\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.018459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.018384    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6f8658b5-ba77-48f1-bb28-79360fe4e918-registry-certificates\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.018459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.018441    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6f8658b5-ba77-48f1-bb28-79360fe4e918-trusted-ca\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.022091 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.022070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6f8658b5-ba77-48f1-bb28-79360fe4e918-image-registry-private-configuration\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.022199 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.022069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6f8658b5-ba77-48f1-bb28-79360fe4e918-installation-pull-secrets\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.022199 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.022076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-registry-tls\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.028036 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.028002    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-bound-sa-token\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.028811 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.028794    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxrd2\" (UniqueName: \"kubernetes.io/projected/6f8658b5-ba77-48f1-bb28-79360fe4e918-kube-api-access-bxrd2\") pod \"image-registry-78c985b499-qf2qg\" (UID: \"6f8658b5-ba77-48f1-bb28-79360fe4e918\") " pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.119363 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f24288a-aa4e-4462-ae64-8cbdbd8c5c42-cert\") pod \"ingress-canary-cncqn\" (UID: \"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42\") " pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.119572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa4203-4fe3-4862-a527-401fb126b252-metrics-tls\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.119572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119412    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4xql\" (UniqueName: \"kubernetes.io/projected/7215b5c0-7b9b-476e-8f79-e46eac80dbca-kube-api-access-r4xql\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.119572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7215b5c0-7b9b-476e-8f79-e46eac80dbca-data-volume\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.119572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpg58\" (UniqueName: \"kubernetes.io/projected/45fa4203-4fe3-4862-a527-401fb126b252-kube-api-access-wpg58\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.119572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7215b5c0-7b9b-476e-8f79-e46eac80dbca-crio-socket\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.119955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215b5c0-7b9b-476e-8f79-e46eac80dbca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.119955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45fa4203-4fe3-4862-a527-401fb126b252-tmp-dir\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.119955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119770    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7215b5c0-7b9b-476e-8f79-e46eac80dbca-crio-socket\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.119955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215b5c0-7b9b-476e-8f79-e46eac80dbca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.119955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45fa4203-4fe3-4862-a527-401fb126b252-config-volume\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.119955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.119840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59zz5\" (UniqueName: \"kubernetes.io/projected/0f24288a-aa4e-4462-ae64-8cbdbd8c5c42-kube-api-access-59zz5\") pod \"ingress-canary-cncqn\" (UID: \"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42\") " pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.120303 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.120074    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45fa4203-4fe3-4862-a527-401fb126b252-tmp-dir\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.120303 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.120265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7215b5c0-7b9b-476e-8f79-e46eac80dbca-data-volume\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.120390 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.120345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45fa4203-4fe3-4862-a527-401fb126b252-config-volume\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.120581 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.120560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7215b5c0-7b9b-476e-8f79-e46eac80dbca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.122085 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.122059    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7215b5c0-7b9b-476e-8f79-e46eac80dbca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.122357 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.122336    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa4203-4fe3-4862-a527-401fb126b252-metrics-tls\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.122407 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.122391    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f24288a-aa4e-4462-ae64-8cbdbd8c5c42-cert\") pod \"ingress-canary-cncqn\" (UID: \"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42\") " pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.149366 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.149328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59zz5\" (UniqueName: \"kubernetes.io/projected/0f24288a-aa4e-4462-ae64-8cbdbd8c5c42-kube-api-access-59zz5\") pod \"ingress-canary-cncqn\" (UID: \"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42\") " pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.151485 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.151455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4xql\" (UniqueName: \"kubernetes.io/projected/7215b5c0-7b9b-476e-8f79-e46eac80dbca-kube-api-access-r4xql\") pod \"insights-runtime-extractor-8h4jc\" (UID: \"7215b5c0-7b9b-476e-8f79-e46eac80dbca\") " pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.156202 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.156172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpg58\" (UniqueName: \"kubernetes.io/projected/45fa4203-4fe3-4862-a527-401fb126b252-kube-api-access-wpg58\") pod \"dns-default-5k478\" (UID: \"45fa4203-4fe3-4862-a527-401fb126b252\") " pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.206751 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.206719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:51.231159 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.231128    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cncqn"
Aug 04 15:11:51.243149 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.243121    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8h4jc"
Aug 04 15:11:51.320326 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.320287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:51.462499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.462464    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cncqn"]
Aug 04 15:11:51.466712 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:51.466682    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f24288a_aa4e_4462_ae64_8cbdbd8c5c42.slice/crio-656a94bae6fea581fa4648ce6bb76982b06aa0c9d0cc02fc277d6ea29b150347 WatchSource:0}: Error finding container 656a94bae6fea581fa4648ce6bb76982b06aa0c9d0cc02fc277d6ea29b150347: Status 404 returned error can't find the container with id 656a94bae6fea581fa4648ce6bb76982b06aa0c9d0cc02fc277d6ea29b150347
Aug 04 15:11:51.488902 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.488865    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8h4jc"]
Aug 04 15:11:51.494446 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:51.494416    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7215b5c0_7b9b_476e_8f79_e46eac80dbca.slice/crio-47a8f4bc8829424a5e7f4174d64591598ad13eae763907937501661618048b63 WatchSource:0}: Error finding container 47a8f4bc8829424a5e7f4174d64591598ad13eae763907937501661618048b63: Status 404 returned error can't find the container with id 47a8f4bc8829424a5e7f4174d64591598ad13eae763907937501661618048b63
Aug 04 15:11:51.498962 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.498560    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78c985b499-qf2qg"]
Aug 04 15:11:51.502992 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:51.502968    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f8658b5_ba77_48f1_bb28_79360fe4e918.slice/crio-679275765ac994d58b2fba127138b08a3d1e8cd92465b7e9e0f34deec0704ade WatchSource:0}: Error finding container 679275765ac994d58b2fba127138b08a3d1e8cd92465b7e9e0f34deec0704ade: Status 404 returned error can't find the container with id 679275765ac994d58b2fba127138b08a3d1e8cd92465b7e9e0f34deec0704ade
Aug 04 15:11:51.531861 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.531832    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5k478"]
Aug 04 15:11:51.541857 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:11:51.541817    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fa4203_4fe3_4862_a527_401fb126b252.slice/crio-836979789ae92b6892ded9defe6b939a29636af3d4bb14272a4bd7662e498fe4 WatchSource:0}: Error finding container 836979789ae92b6892ded9defe6b939a29636af3d4bb14272a4bd7662e498fe4: Status 404 returned error can't find the container with id 836979789ae92b6892ded9defe6b939a29636af3d4bb14272a4bd7662e498fe4
Aug 04 15:11:51.959108 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.959070    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:11:51.959108 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.959100    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:11:51.959600 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.959473    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:11:51.962948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.962860    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 15:11:51.963074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.962958    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 15:11:51.963127 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.963108    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 15:11:51.963223 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.963201    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7r4m\""
Aug 04 15:11:51.963293 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.962861    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k8gfq\""
Aug 04 15:11:51.963328 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:51.962869    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 15:11:52.180264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.180217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8h4jc" event={"ID":"7215b5c0-7b9b-476e-8f79-e46eac80dbca","Type":"ContainerStarted","Data":"4c10f2f3eb1f662b9269624dfec8216d83dd7cadd8860c8bf91e94d02dfc95b6"}
Aug 04 15:11:52.180724 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.180327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8h4jc" event={"ID":"7215b5c0-7b9b-476e-8f79-e46eac80dbca","Type":"ContainerStarted","Data":"47a8f4bc8829424a5e7f4174d64591598ad13eae763907937501661618048b63"}
Aug 04 15:11:52.181461 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.181430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5k478" event={"ID":"45fa4203-4fe3-4862-a527-401fb126b252","Type":"ContainerStarted","Data":"836979789ae92b6892ded9defe6b939a29636af3d4bb14272a4bd7662e498fe4"}
Aug 04 15:11:52.182591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.182533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cncqn" event={"ID":"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42","Type":"ContainerStarted","Data":"656a94bae6fea581fa4648ce6bb76982b06aa0c9d0cc02fc277d6ea29b150347"}
Aug 04 15:11:52.184172 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.184150    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c985b499-qf2qg" event={"ID":"6f8658b5-ba77-48f1-bb28-79360fe4e918","Type":"ContainerStarted","Data":"150c559c29cee4c0c1739ad0416e8eb4184f16e667ec983abc580642062c3095"}
Aug 04 15:11:52.184287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.184193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78c985b499-qf2qg" event={"ID":"6f8658b5-ba77-48f1-bb28-79360fe4e918","Type":"ContainerStarted","Data":"679275765ac994d58b2fba127138b08a3d1e8cd92465b7e9e0f34deec0704ade"}
Aug 04 15:11:52.184355 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.184326    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:11:52.216866 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:52.216757    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78c985b499-qf2qg" podStartSLOduration=2.216739205 podStartE2EDuration="2.216739205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:11:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:52.21408827 +0000 UTC m=+54.720386326" watchObservedRunningTime="2026-08-04 15:11:52.216739205 +0000 UTC m=+54.723037260"
Aug 04 15:11:55.195226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.195187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8h4jc" event={"ID":"7215b5c0-7b9b-476e-8f79-e46eac80dbca","Type":"ContainerStarted","Data":"bb44e620676d98962825d7924b33571eb132476ca9782ae6e86a706ab73f99a1"}
Aug 04 15:11:55.196997 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.196968    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5k478" event={"ID":"45fa4203-4fe3-4862-a527-401fb126b252","Type":"ContainerStarted","Data":"3325e20f39aff95dd8831b7bf91a55c37f73184f0cd8c4eb695de47befe16dc0"}
Aug 04 15:11:55.197083 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.197017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5k478" event={"ID":"45fa4203-4fe3-4862-a527-401fb126b252","Type":"ContainerStarted","Data":"e4308e24a5e38ecca1b80cbb7310bbbfae9045398d211cd2615bab7934bca5b4"}
Aug 04 15:11:55.197083 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.197039    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5k478"
Aug 04 15:11:55.198491 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.198465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cncqn" event={"ID":"0f24288a-aa4e-4462-ae64-8cbdbd8c5c42","Type":"ContainerStarted","Data":"f65acac109a174f210248f56c0da2a5b77f89444e11c5fc8c29b10c2e2c1f01a"}
Aug 04 15:11:55.224739 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.222159    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5k478" podStartSLOduration=2.438177041 podStartE2EDuration="5.222136818s" totalImagesPullingTime="2.783959777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:55.220252613 +0000 UTC m=+57.726550670" watchObservedRunningTime="2026-08-04 15:11:55.222136818 +0000 UTC m=+57.728434874"
Aug 04 15:11:55.242903 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:55.242843    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cncqn" podStartSLOduration=2.383229294 podStartE2EDuration="5.242822556s" totalImagesPullingTime="2.859593262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:55.241409937 +0000 UTC m=+57.747707993" watchObservedRunningTime="2026-08-04 15:11:55.242822556 +0000 UTC m=+57.749120612"
Aug 04 15:11:57.206653 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:57.206604    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8h4jc" event={"ID":"7215b5c0-7b9b-476e-8f79-e46eac80dbca","Type":"ContainerStarted","Data":"2ecc61cb8c190f1e5aec2166569f86a226932c73db31a2ff3d7306fee77580a5"}
Aug 04 15:11:57.252343 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:57.252285    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8h4jc" podStartSLOduration=2.176746073 podStartE2EDuration="7.252266218s" totalImagesPullingTime="5.075520145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:11:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:57.25068969 +0000 UTC m=+59.756987743" watchObservedRunningTime="2026-08-04 15:11:57.252266218 +0000 UTC m=+59.758564301"
Aug 04 15:11:58.128389 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:11:58.128358    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dlwvs"
Aug 04 15:12:03.722852 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.722802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:12:03.722852 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.722854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:12:03.727151 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.727127    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 04 15:12:03.727449 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.727433    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 04 15:12:03.735925 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.735892    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f8b016ed-b95c-4ffc-897b-478f53ba0203-metrics-certs\") pod \"network-metrics-daemon-vj74l\" (UID: \"f8b016ed-b95c-4ffc-897b-478f53ba0203\") " pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:12:03.736044 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.735904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/319e5610-9a08-4abb-b819-a1d286392a65-original-pull-secret\") pod \"global-pull-secret-syncer-bzd8c\" (UID: \"319e5610-9a08-4abb-b819-a1d286392a65\") " pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:12:03.823430 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.823380    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:12:03.828150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.828126    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 04 15:12:03.837069 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.837044    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 04 15:12:03.847974 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.847947    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxh9\" (UniqueName: \"kubernetes.io/projected/0a5095ac-244f-48c3-90f8-de9a5a127ab8-kube-api-access-2jxh9\") pod \"network-check-target-tc5l7\" (UID: \"0a5095ac-244f-48c3-90f8-de9a5a127ab8\") " pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:12:03.977448 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.977359    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k8gfq\""
Aug 04 15:12:03.982029 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.982006    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzd8c"
Aug 04 15:12:03.985133 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.985114    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vj74l"
Aug 04 15:12:03.992181 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.992162    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7r4m\""
Aug 04 15:12:03.999354 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:03.999319    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:12:04.147411 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:04.147370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vj74l"]
Aug 04 15:12:04.150327 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:04.150298    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b016ed_b95c_4ffc_897b_478f53ba0203.slice/crio-f98d77117a73744f58eaadfbc08ef87c7ad2b789fbbf2b2cc3aa8b6436955c8f WatchSource:0}: Error finding container f98d77117a73744f58eaadfbc08ef87c7ad2b789fbbf2b2cc3aa8b6436955c8f: Status 404 returned error can't find the container with id f98d77117a73744f58eaadfbc08ef87c7ad2b789fbbf2b2cc3aa8b6436955c8f
Aug 04 15:12:04.228397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:04.228307    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vj74l" event={"ID":"f8b016ed-b95c-4ffc-897b-478f53ba0203","Type":"ContainerStarted","Data":"f98d77117a73744f58eaadfbc08ef87c7ad2b789fbbf2b2cc3aa8b6436955c8f"}
Aug 04 15:12:04.372206 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:04.372164    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bzd8c"]
Aug 04 15:12:04.372883 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:04.372862    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tc5l7"]
Aug 04 15:12:04.375107 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:04.375078    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319e5610_9a08_4abb_b819_a1d286392a65.slice/crio-bc08cae49984ae0c5e73cb9be82e14baab134df41e9657649707d9d5a9483968 WatchSource:0}: Error finding container bc08cae49984ae0c5e73cb9be82e14baab134df41e9657649707d9d5a9483968: Status 404 returned error can't find the container with id bc08cae49984ae0c5e73cb9be82e14baab134df41e9657649707d9d5a9483968
Aug 04 15:12:04.375662 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:04.375615    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5095ac_244f_48c3_90f8_de9a5a127ab8.slice/crio-0c88697cba4d83cec0f01e1efe90cd185fd752ecc722c05b38ba580a9d6daf52 WatchSource:0}: Error finding container 0c88697cba4d83cec0f01e1efe90cd185fd752ecc722c05b38ba580a9d6daf52: Status 404 returned error can't find the container with id 0c88697cba4d83cec0f01e1efe90cd185fd752ecc722c05b38ba580a9d6daf52
Aug 04 15:12:05.204823 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:05.204793    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5k478"
Aug 04 15:12:05.234317 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:05.234286    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bzd8c" event={"ID":"319e5610-9a08-4abb-b819-a1d286392a65","Type":"ContainerStarted","Data":"bc08cae49984ae0c5e73cb9be82e14baab134df41e9657649707d9d5a9483968"}
Aug 04 15:12:05.235792 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:05.235758    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tc5l7" event={"ID":"0a5095ac-244f-48c3-90f8-de9a5a127ab8","Type":"ContainerStarted","Data":"0c88697cba4d83cec0f01e1efe90cd185fd752ecc722c05b38ba580a9d6daf52"}
Aug 04 15:12:06.241964 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:06.241865    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vj74l" event={"ID":"f8b016ed-b95c-4ffc-897b-478f53ba0203","Type":"ContainerStarted","Data":"9a83b1f324854ae89ba2f090176340275905338c9ac04ce96a6f5aaaec324a81"}
Aug 04 15:12:06.241964 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:06.241944    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vj74l" event={"ID":"f8b016ed-b95c-4ffc-897b-478f53ba0203","Type":"ContainerStarted","Data":"5f24c7c277a49d6d847efc735eb458dc89f4975c86e66fd13c6a8fa1434b3079"}
Aug 04 15:12:06.277049 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:06.276985    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vj74l" podStartSLOduration=66.786351306 podStartE2EDuration="1m8.276960884s" totalImagesPullingTime="1.490609578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:06.275333071 +0000 UTC m=+68.781631127" watchObservedRunningTime="2026-08-04 15:12:06.276960884 +0000 UTC m=+68.783258940"
Aug 04 15:12:08.625218 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.625180    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"]
Aug 04 15:12:08.628605 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.628582    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.632266 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.632234    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 04 15:12:08.633433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.633277    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Aug 04 15:12:08.633433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.633296    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Aug 04 15:12:08.633433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.633294    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 04 15:12:08.633433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.633290    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Aug 04 15:12:08.633433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.633389    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 04 15:12:08.633433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.633409    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-g86rf\""
Aug 04 15:12:08.646031 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.645991    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"]
Aug 04 15:12:08.669163 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.669135    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7cwbk"]
Aug 04 15:12:08.672273 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.672254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.675683 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.675655    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 04 15:12:08.675814 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.675687    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qbrvb\""
Aug 04 15:12:08.675814 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.675766    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 04 15:12:08.678771 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.678752    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 04 15:12:08.763227 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-sys\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab61273d-1d70-4fa6-8205-3d5ce36a8744-metrics-client-ca\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763263    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1e8a259-7a35-4a92-8847-14d17a528111-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.763420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.763420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-textfile\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.763420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.763702 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763702 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgc6m\" (UniqueName: \"kubernetes.io/projected/ab61273d-1d70-4fa6-8205-3d5ce36a8744-kube-api-access-pgc6m\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763702 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wrtw\" (UniqueName: \"kubernetes.io/projected/b1e8a259-7a35-4a92-8847-14d17a528111-kube-api-access-9wrtw\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.763702 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763613    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-wtmp\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763702 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-root\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763914 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763711    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-tls\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763914 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-accelerators-collector-config\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.763914 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.763781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b1e8a259-7a35-4a92-8847-14d17a528111-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.864576 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.864576 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-textfile\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864604    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgc6m\" (UniqueName: \"kubernetes.io/projected/ab61273d-1d70-4fa6-8205-3d5ce36a8744-kube-api-access-pgc6m\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wrtw\" (UniqueName: \"kubernetes.io/projected/b1e8a259-7a35-4a92-8847-14d17a528111-kube-api-access-9wrtw\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-wtmp\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-root\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.864846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-tls\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-accelerators-collector-config\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b1e8a259-7a35-4a92-8847-14d17a528111-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.865229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864912    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-sys\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864948    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab61273d-1d70-4fa6-8205-3d5ce36a8744-metrics-client-ca\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.864971    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1e8a259-7a35-4a92-8847-14d17a528111-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.865466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865442    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-root\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865632 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865607    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-wtmp\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865758    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-accelerators-collector-config\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.865962 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-textfile\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.865962 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1e8a259-7a35-4a92-8847-14d17a528111-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.865962 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.865902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab61273d-1d70-4fa6-8205-3d5ce36a8744-sys\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.866105 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:12:08.865967    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Aug 04 15:12:08.866105 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:12:08.866028    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-tls podName:ab61273d-1d70-4fa6-8205-3d5ce36a8744 nodeName:}" failed. No retries permitted until 2026-08-04 15:12:09.366009606 +0000 UTC m=+71.872307644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-tls") pod "node-exporter-7cwbk" (UID: "ab61273d-1d70-4fa6-8205-3d5ce36a8744") : secret "node-exporter-tls" not found
Aug 04 15:12:08.866105 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.866090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b1e8a259-7a35-4a92-8847-14d17a528111-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.866397 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.866372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab61273d-1d70-4fa6-8205-3d5ce36a8744-metrics-client-ca\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.868234 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.868187    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.868413 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.868363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.868622 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.868598    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b1e8a259-7a35-4a92-8847-14d17a528111-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.876491 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.876421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wrtw\" (UniqueName: \"kubernetes.io/projected/b1e8a259-7a35-4a92-8847-14d17a528111-kube-api-access-9wrtw\") pod \"kube-state-metrics-6bd9664c4-2qrq8\" (UID: \"b1e8a259-7a35-4a92-8847-14d17a528111\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:08.880670 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.880630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgc6m\" (UniqueName: \"kubernetes.io/projected/ab61273d-1d70-4fa6-8205-3d5ce36a8744-kube-api-access-pgc6m\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:08.939523 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:08.939491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"
Aug 04 15:12:09.224932 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.224898    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8"]
Aug 04 15:12:09.228863 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:09.228837    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e8a259_7a35_4a92_8847_14d17a528111.slice/crio-e2ef0c7f7edfedc1ad563cc395d95b697938c2194a613a1b559c095622a57af4 WatchSource:0}: Error finding container e2ef0c7f7edfedc1ad563cc395d95b697938c2194a613a1b559c095622a57af4: Status 404 returned error can't find the container with id e2ef0c7f7edfedc1ad563cc395d95b697938c2194a613a1b559c095622a57af4
Aug 04 15:12:09.254608 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.254569    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bzd8c" event={"ID":"319e5610-9a08-4abb-b819-a1d286392a65","Type":"ContainerStarted","Data":"911461dbe3861f2a28ba1415edaa9cac60e372616b8fdd013fcd2a522d449c34"}
Aug 04 15:12:09.255950 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.255916    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8" event={"ID":"b1e8a259-7a35-4a92-8847-14d17a528111","Type":"ContainerStarted","Data":"e2ef0c7f7edfedc1ad563cc395d95b697938c2194a613a1b559c095622a57af4"}
Aug 04 15:12:09.257436 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.257411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tc5l7" event={"ID":"0a5095ac-244f-48c3-90f8-de9a5a127ab8","Type":"ContainerStarted","Data":"54df20f54b8718904be2a091db60d98748ea754a636bf17a9be5416188e100ac"}
Aug 04 15:12:09.257543 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.257461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:12:09.282854 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.282801    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bzd8c" podStartSLOduration=65.602345543 podStartE2EDuration="1m10.282785398s" totalImagesPullingTime="4.680439855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:09.282126365 +0000 UTC m=+71.788424420" watchObservedRunningTime="2026-08-04 15:12:09.282785398 +0000 UTC m=+71.789083444"
Aug 04 15:12:09.302559 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.302505    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tc5l7" podStartSLOduration=65.63507778 podStartE2EDuration="1m10.302487255s" totalImagesPullingTime="4.667409475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:09.301860178 +0000 UTC m=+71.808158234" watchObservedRunningTime="2026-08-04 15:12:09.302487255 +0000 UTC m=+71.808785310"
Aug 04 15:12:09.370428 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.370388    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-tls\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:09.372879 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.372847    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ab61273d-1d70-4fa6-8205-3d5ce36a8744-node-exporter-tls\") pod \"node-exporter-7cwbk\" (UID: \"ab61273d-1d70-4fa6-8205-3d5ce36a8744\") " pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:09.465200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.465121    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6598684769-k4t8g"]
Aug 04 15:12:09.468437 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.468412    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.474948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.474805    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 04 15:12:09.474948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.474835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 04 15:12:09.474948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.474906    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 04 15:12:09.475145 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.475114    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lzz5b\""
Aug 04 15:12:09.475145 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.475125    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 04 15:12:09.475227 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.475212    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 04 15:12:09.475788 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.475770    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 04 15:12:09.476084 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.476070    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 04 15:12:09.487549 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.486683    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6598684769-k4t8g"]
Aug 04 15:12:09.572410 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.572365    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rqqv\" (UniqueName: \"kubernetes.io/projected/35546526-2227-47ec-aca2-2be57733619b-kube-api-access-4rqqv\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.572410 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.572410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-service-ca\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.572698 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.572446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-console-config\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.572698 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.572546    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-serving-cert\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.572698 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.572597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-oauth-serving-cert\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.572698 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.572664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-oauth-config\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.583437 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.583405    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7cwbk"
Aug 04 15:12:09.593396 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:09.593368    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab61273d_1d70_4fa6_8205_3d5ce36a8744.slice/crio-181241fc3d23b475cf9f393c5998596fa76a621db0f16798b02d116ee6aab457 WatchSource:0}: Error finding container 181241fc3d23b475cf9f393c5998596fa76a621db0f16798b02d116ee6aab457: Status 404 returned error can't find the container with id 181241fc3d23b475cf9f393c5998596fa76a621db0f16798b02d116ee6aab457
Aug 04 15:12:09.673991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.673952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-serving-cert\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674452 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-oauth-serving-cert\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674452 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674038    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-oauth-config\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674452 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rqqv\" (UniqueName: \"kubernetes.io/projected/35546526-2227-47ec-aca2-2be57733619b-kube-api-access-4rqqv\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674452 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-service-ca\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674452 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-console-config\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674904 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674879    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-service-ca\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.674904 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674894    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-console-config\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.675037 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.674920    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-oauth-serving-cert\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.676548 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.676527    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-oauth-config\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.676663 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.676548    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-serving-cert\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.690940 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.690918    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rqqv\" (UniqueName: \"kubernetes.io/projected/35546526-2227-47ec-aca2-2be57733619b-kube-api-access-4rqqv\") pod \"console-6598684769-k4t8g\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") " pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.777784 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.777686    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:09.929575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.929532    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6598684769-k4t8g"]
Aug 04 15:12:09.930521 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.930501    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:12:09.932459 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:09.932427    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35546526_2227_47ec_aca2_2be57733619b.slice/crio-b65e3da9854a161f1c64297c22ca1d5cf4583f3f7c04a266135e0a6433ad4ea3 WatchSource:0}: Error finding container b65e3da9854a161f1c64297c22ca1d5cf4583f3f7c04a266135e0a6433ad4ea3: Status 404 returned error can't find the container with id b65e3da9854a161f1c64297c22ca1d5cf4583f3f7c04a266135e0a6433ad4ea3
Aug 04 15:12:09.935288 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.935269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:09.939463 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.939442    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 04 15:12:09.941201 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941183    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 04 15:12:09.941480 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941451    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 04 15:12:09.941480 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941475    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 04 15:12:09.941678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941521    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 04 15:12:09.941678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941548    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 04 15:12:09.941678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941577    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 04 15:12:09.941678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941586    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-m5lvm\""
Aug 04 15:12:09.941843 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.941830    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 04 15:12:09.965750 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:09.965719    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 04 15:12:10.075240 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.075115    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:12:10.077012 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.076991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077048    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjr7\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-kube-api-access-vnjr7\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077135    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077220 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-out\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077220 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077220 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077220 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077348 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-web-config\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077348 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077348 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.077447 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.077361    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178328 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178290    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178526 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178526 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjr7\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-kube-api-access-vnjr7\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178526 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178393    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178526 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178526 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-out\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178810 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178869 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178869 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178860    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178911    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-web-config\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.178977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178948    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.179069 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.178982    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.179069 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.179035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.179250 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.179226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.180575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.180548    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.182210 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.181369    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.182210 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.181728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-volume\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.182210 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.181774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-out\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.184888 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.182829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.184888 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.183395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.184888 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.183530    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.184888 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.183957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.184888 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.184401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.185357 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.185333    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-web-config\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.185489 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.185465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.201917 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.201881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjr7\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-kube-api-access-vnjr7\") pod \"alertmanager-main-0\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.246311 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.246268    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:12:10.261834 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.261798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598684769-k4t8g" event={"ID":"35546526-2227-47ec-aca2-2be57733619b","Type":"ContainerStarted","Data":"b65e3da9854a161f1c64297c22ca1d5cf4583f3f7c04a266135e0a6433ad4ea3"}
Aug 04 15:12:10.263078 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.263044    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7cwbk" event={"ID":"ab61273d-1d70-4fa6-8205-3d5ce36a8744","Type":"ContainerStarted","Data":"181241fc3d23b475cf9f393c5998596fa76a621db0f16798b02d116ee6aab457"}
Aug 04 15:12:10.426047 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:10.426017    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:12:10.602934 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:10.602852    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcf31c6_a871_4584_aff1_89cbab0b99c4.slice/crio-0dcf246d77876d00b0d79582c2cb829510430231e4472bffb4e10efee6b8ae4b WatchSource:0}: Error finding container 0dcf246d77876d00b0d79582c2cb829510430231e4472bffb4e10efee6b8ae4b: Status 404 returned error can't find the container with id 0dcf246d77876d00b0d79582c2cb829510430231e4472bffb4e10efee6b8ae4b
Aug 04 15:12:11.269756 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:11.269619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"0dcf246d77876d00b0d79582c2cb829510430231e4472bffb4e10efee6b8ae4b"}
Aug 04 15:12:11.272246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:11.272122    2569 generic.go:356] "Generic (PLEG): container finished" podID="ab61273d-1d70-4fa6-8205-3d5ce36a8744" containerID="9b312542380c3da3ed9cdccb475cd39da61f5c6f0e2e9b91666cd3db8d8a4f97" exitCode=0
Aug 04 15:12:11.272246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:11.272186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7cwbk" event={"ID":"ab61273d-1d70-4fa6-8205-3d5ce36a8744","Type":"ContainerDied","Data":"9b312542380c3da3ed9cdccb475cd39da61f5c6f0e2e9b91666cd3db8d8a4f97"}
Aug 04 15:12:12.278927 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:12.278889    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7cwbk" event={"ID":"ab61273d-1d70-4fa6-8205-3d5ce36a8744","Type":"ContainerStarted","Data":"60ee57623415296170f70c8b938fd39440c434b1fb2eb626be4442ab02c10c67"}
Aug 04 15:12:12.279495 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:12.278953    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7cwbk" event={"ID":"ab61273d-1d70-4fa6-8205-3d5ce36a8744","Type":"ContainerStarted","Data":"58b16a7e598d8a1d275e8d54af61bf522ed38b6ef896bedc79ba9552aeafd62b"}
Aug 04 15:12:12.332725 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:12.332661    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7cwbk" podStartSLOduration=3.215848951 podStartE2EDuration="4.332625941s" totalImagesPullingTime="1.05177487s" totalInitContainerRuntime="65.00212ms" isStatefulPod=true podCreationTimestamp="2026-08-04 15:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:12.33191393 +0000 UTC m=+74.838211986" watchObservedRunningTime="2026-08-04 15:12:12.332625941 +0000 UTC m=+74.838924011"
Aug 04 15:12:13.192228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:13.192197    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78c985b499-qf2qg"
Aug 04 15:12:13.283720 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:13.283684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598684769-k4t8g" event={"ID":"35546526-2227-47ec-aca2-2be57733619b","Type":"ContainerStarted","Data":"b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186"}
Aug 04 15:12:13.318892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:13.318833    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6598684769-k4t8g" podStartSLOduration=1.048408657 podStartE2EDuration="4.318813309s" totalImagesPullingTime="3.270404652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:13.317069504 +0000 UTC m=+75.823367559" watchObservedRunningTime="2026-08-04 15:12:13.318813309 +0000 UTC m=+75.825111403"
Aug 04 15:12:14.291289 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:14.291257    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8" event={"ID":"b1e8a259-7a35-4a92-8847-14d17a528111","Type":"ContainerStarted","Data":"eb0a42a08d7a2bd4966eddf60518574802fb336de4a07aae2514c015b6a81511"}
Aug 04 15:12:14.291710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:14.291334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8" event={"ID":"b1e8a259-7a35-4a92-8847-14d17a528111","Type":"ContainerStarted","Data":"18279c425fd41d08bddd930a66a86074b063d500140867f272e5cd13d4a4f26e"}
Aug 04 15:12:15.071130 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.071045    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:12:15.074689 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.074670    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.078246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.078217    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 04 15:12:15.082025 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.081999    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 04 15:12:15.082171 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.082059    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 04 15:12:15.082262 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.082242    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 04 15:12:15.082384 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.082370    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 04 15:12:15.083276 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083254    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 04 15:12:15.083276 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083264    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 04 15:12:15.083477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083279    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 04 15:12:15.083477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083279    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 04 15:12:15.083477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083429    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-q8ts6\""
Aug 04 15:12:15.083616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083556    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 04 15:12:15.083616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083556    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 04 15:12:15.083736 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083700    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 04 15:12:15.083815 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083802    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 04 15:12:15.083863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083816    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b3rl1aiq3n1cg\""
Aug 04 15:12:15.083863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.083827    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 04 15:12:15.084725 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.084711    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 04 15:12:15.104234 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.104203    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:12:15.227198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227164    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227229    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227254    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227326    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227382    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227428    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227528    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227662    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxjpg\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-kube-api-access-nxjpg\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227756    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.227935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.227808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.296288 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.296252    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea" exitCode=0
Aug 04 15:12:15.296757 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.296337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"}
Aug 04 15:12:15.298748 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.298712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8" event={"ID":"b1e8a259-7a35-4a92-8847-14d17a528111","Type":"ContainerStarted","Data":"e7b4926739e35e088227be89634a8429698ea6a545ef71dd4d9b62cdbac63340"}
Aug 04 15:12:15.329138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329088    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxjpg\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-kube-api-access-nxjpg\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329105    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329179    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329223    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.329459 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329581    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329613    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.329856    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330319 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.330035    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330319 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.330137    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.330319 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.330146    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.331241 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.330415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.331241 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.330799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.331438 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.331382    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.332968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.333524    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.333752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334054    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334223    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config-out\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334223    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334568    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.334973 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.334928    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.336471 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.336445    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.336583 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.336553    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-web-config\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.344222 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.344195    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxjpg\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-kube-api-access-nxjpg\") pod \"prometheus-k8s-0\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.351600 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.351548    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-2qrq8" podStartSLOduration=2.448715174 podStartE2EDuration="7.351529516s" totalImagesPullingTime="4.902814342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:15.350233075 +0000 UTC m=+77.856531137" watchObservedRunningTime="2026-08-04 15:12:15.351529516 +0000 UTC m=+77.857827616"
Aug 04 15:12:15.384867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.384815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:15.552945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:15.552919    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:12:15.554562 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:12:15.554519    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822c63a7_acbc_4e8b_9b01_a7ef9fea43a8.slice/crio-d5a1ee3d5c910a2338e41df034e9445ba3402dee07864b0cded812f131b2e8c8 WatchSource:0}: Error finding container d5a1ee3d5c910a2338e41df034e9445ba3402dee07864b0cded812f131b2e8c8: Status 404 returned error can't find the container with id d5a1ee3d5c910a2338e41df034e9445ba3402dee07864b0cded812f131b2e8c8
Aug 04 15:12:16.304431 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:16.304392    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f" exitCode=0
Aug 04 15:12:16.304884 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:16.304477    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"}
Aug 04 15:12:16.304884 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:16.304548    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"d5a1ee3d5c910a2338e41df034e9445ba3402dee07864b0cded812f131b2e8c8"}
Aug 04 15:12:17.319415 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:17.319376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"}
Aug 04 15:12:17.319822 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:17.319469    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"}
Aug 04 15:12:17.319822 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:17.319510    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"}
Aug 04 15:12:17.319822 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:17.319551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"}
Aug 04 15:12:18.328883 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:18.328795    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"}
Aug 04 15:12:18.329311 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:18.328917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerStarted","Data":"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"}
Aug 04 15:12:18.370992 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:18.370927    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.898233565 podStartE2EDuration="9.370910254s" totalImagesPullingTime="6.455662446s" totalInitContainerRuntime="17.014243ms" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:18.369518083 +0000 UTC m=+80.875816149" watchObservedRunningTime="2026-08-04 15:12:18.370910254 +0000 UTC m=+80.877208309"
Aug 04 15:12:19.778141 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:19.778111    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:19.778522 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:19.778179    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:19.784368 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:19.784339    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:20.339743 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:20.339706    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"}
Aug 04 15:12:20.339929 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:20.339777    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"}
Aug 04 15:12:20.343748 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:20.343726    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:12:22.356606 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:22.356507    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"}
Aug 04 15:12:22.356606 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:22.356573    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"}
Aug 04 15:12:22.356606 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:22.356599    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"}
Aug 04 15:12:22.357118 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:22.356657    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerStarted","Data":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"}
Aug 04 15:12:22.390015 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:22.389953    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.198945483 podStartE2EDuration="7.38992982s" totalImagesPullingTime="5.170693463s" totalInitContainerRuntime="20.290874ms" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:22.388918763 +0000 UTC m=+84.895216833" watchObservedRunningTime="2026-08-04 15:12:22.38992982 +0000 UTC m=+84.896227877"
Aug 04 15:12:25.385956 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:25.385917    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:12:40.265863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:40.265829    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tc5l7"
Aug 04 15:12:40.402972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:12:40.402930    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6598684769-k4t8g"]
Aug 04 15:13:05.424969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.424899    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6598684769-k4t8g" podUID="35546526-2227-47ec-aca2-2be57733619b" containerName="console" containerID="cri-o://b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186" gracePeriod=15
Aug 04 15:13:05.673024 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.672996    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:13:05.751034 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.750958    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-serving-cert\") pod \"35546526-2227-47ec-aca2-2be57733619b\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") "
Aug 04 15:13:05.751182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751041    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-oauth-config\") pod \"35546526-2227-47ec-aca2-2be57733619b\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") "
Aug 04 15:13:05.751182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751078    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-console-config\") pod \"35546526-2227-47ec-aca2-2be57733619b\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") "
Aug 04 15:13:05.751182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751118    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-service-ca\") pod \"35546526-2227-47ec-aca2-2be57733619b\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") "
Aug 04 15:13:05.751182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751156    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-oauth-serving-cert\") pod \"35546526-2227-47ec-aca2-2be57733619b\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") "
Aug 04 15:13:05.751372 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rqqv\" (UniqueName: \"kubernetes.io/projected/35546526-2227-47ec-aca2-2be57733619b-kube-api-access-4rqqv\") pod \"35546526-2227-47ec-aca2-2be57733619b\" (UID: \"35546526-2227-47ec-aca2-2be57733619b\") "
Aug 04 15:13:05.751579 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751545    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-service-ca" (OuterVolumeSpecName: "service-ca") pod "35546526-2227-47ec-aca2-2be57733619b" (UID: "35546526-2227-47ec-aca2-2be57733619b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:05.751753 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751568    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-console-config" (OuterVolumeSpecName: "console-config") pod "35546526-2227-47ec-aca2-2be57733619b" (UID: "35546526-2227-47ec-aca2-2be57733619b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:05.751753 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.751582    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "35546526-2227-47ec-aca2-2be57733619b" (UID: "35546526-2227-47ec-aca2-2be57733619b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:05.753441 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.753409    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35546526-2227-47ec-aca2-2be57733619b-kube-api-access-4rqqv" (OuterVolumeSpecName: "kube-api-access-4rqqv") pod "35546526-2227-47ec-aca2-2be57733619b" (UID: "35546526-2227-47ec-aca2-2be57733619b"). InnerVolumeSpecName "kube-api-access-4rqqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:13:05.753441 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.753428    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "35546526-2227-47ec-aca2-2be57733619b" (UID: "35546526-2227-47ec-aca2-2be57733619b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:05.753616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.753496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "35546526-2227-47ec-aca2-2be57733619b" (UID: "35546526-2227-47ec-aca2-2be57733619b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:05.852785 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.852744    2569 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:05.852785 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.852777    2569 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/35546526-2227-47ec-aca2-2be57733619b-console-oauth-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:05.852785 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.852787    2569 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-console-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:05.852785 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.852795    2569 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:05.853050 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.852804    2569 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/35546526-2227-47ec-aca2-2be57733619b-oauth-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:05.853050 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:05.852814    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rqqv\" (UniqueName: \"kubernetes.io/projected/35546526-2227-47ec-aca2-2be57733619b-kube-api-access-4rqqv\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:06.518322 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.518284    2569 generic.go:356] "Generic (PLEG): container finished" podID="35546526-2227-47ec-aca2-2be57733619b" containerID="b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186" exitCode=0
Aug 04 15:13:06.518760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.518334    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6598684769-k4t8g"
Aug 04 15:13:06.518760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.518375    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598684769-k4t8g" event={"ID":"35546526-2227-47ec-aca2-2be57733619b","Type":"ContainerDied","Data":"b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186"}
Aug 04 15:13:06.518760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.518433    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6598684769-k4t8g" event={"ID":"35546526-2227-47ec-aca2-2be57733619b","Type":"ContainerDied","Data":"b65e3da9854a161f1c64297c22ca1d5cf4583f3f7c04a266135e0a6433ad4ea3"}
Aug 04 15:13:06.518760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.518458    2569 scope.go:117] "RemoveContainer" containerID="b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186"
Aug 04 15:13:06.526773 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.526754    2569 scope.go:117] "RemoveContainer" containerID="b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186"
Aug 04 15:13:06.527080 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:06.527050    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186\": container with ID starting with b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186 not found: ID does not exist" containerID="b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186"
Aug 04 15:13:06.527162 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.527094    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186"} err="failed to get container status \"b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186\": rpc error: code = NotFound desc = could not find container \"b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186\": container with ID starting with b866c9530f8818c111abd6740d3a582efcd23914dc8cb3b6e4872849e278c186 not found: ID does not exist"
Aug 04 15:13:06.545536 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.545489    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6598684769-k4t8g"]
Aug 04 15:13:06.552384 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:06.552350    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6598684769-k4t8g"]
Aug 04 15:13:07.963332 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:07.963294    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35546526-2227-47ec-aca2-2be57733619b" path="/var/lib/kubelet/pods/35546526-2227-47ec-aca2-2be57733619b/volumes"
Aug 04 15:13:15.385187 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:15.385137    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:15.405991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:15.405957    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:15.567242 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:15.567214    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:29.199125 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199082    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:13:29.199580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199381    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="alertmanager" containerID="cri-o://a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd" gracePeriod=120
Aug 04 15:13:29.199580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199447    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-metric" containerID="cri-o://b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c" gracePeriod=120
Aug 04 15:13:29.199580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199499    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy" containerID="cri-o://7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398" gracePeriod=120
Aug 04 15:13:29.199580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199509    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="config-reloader" containerID="cri-o://96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31" gracePeriod=120
Aug 04 15:13:29.199580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199548    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="prom-label-proxy" containerID="cri-o://9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856" gracePeriod=120
Aug 04 15:13:29.199900 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.199475    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-web" containerID="cri-o://c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5" gracePeriod=120
Aug 04 15:13:29.604073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.603983    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856" exitCode=0
Aug 04 15:13:29.604073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604007    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398" exitCode=0
Aug 04 15:13:29.604073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604013    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31" exitCode=0
Aug 04 15:13:29.604073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604019    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd" exitCode=0
Aug 04 15:13:29.604073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604057    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"}
Aug 04 15:13:29.604341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604121    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"}
Aug 04 15:13:29.604341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604148    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"}
Aug 04 15:13:29.604341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:29.604173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"}
Aug 04 15:13:30.442253 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.442228    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.563703 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563609    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-out\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.563703 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563668    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-trusted-ca-bundle\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.563920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563714    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-web-config\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.563920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563745    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-cluster-tls-config\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.563920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563785    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.563920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563824    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.563920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.563869    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnjr7\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-kube-api-access-vnjr7\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.564177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564079    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:30.564177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564091    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-tls-assets\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.564177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564143    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-main-tls\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.564353 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564179    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-metrics-client-ca\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.564353 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564235    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-volume\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.564353 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564280    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-web\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.564353 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564306    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-main-db\") pod \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\" (UID: \"1fcf31c6-a871-4584-aff1-89cbab0b99c4\") "
Aug 04 15:13:30.565687 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.564583    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.565687 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.565210    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:30.565687 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.565619    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:13:30.566531 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.566504    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-kube-api-access-vnjr7" (OuterVolumeSpecName: "kube-api-access-vnjr7") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "kube-api-access-vnjr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:13:30.567424 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.567387    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-out" (OuterVolumeSpecName: "config-out") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:13:30.567757 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.567723    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.568392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.568354    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.568502 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.568476    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-volume" (OuterVolumeSpecName: "config-volume") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.568792 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.568771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:13:30.568910 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.568892    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.570304 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.570277    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.572200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.572173    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.577402 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.577372    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-web-config" (OuterVolumeSpecName: "web-config") pod "1fcf31c6-a871-4584-aff1-89cbab0b99c4" (UID: "1fcf31c6-a871-4584-aff1-89cbab0b99c4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:30.610461 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610421    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c" exitCode=0
Aug 04 15:13:30.610461 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610450    2569 generic.go:356] "Generic (PLEG): container finished" podID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerID="c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5" exitCode=0
Aug 04 15:13:30.610731 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610476    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"}
Aug 04 15:13:30.610731 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610509    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.610731 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"}
Aug 04 15:13:30.610731 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610591    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1fcf31c6-a871-4584-aff1-89cbab0b99c4","Type":"ContainerDied","Data":"0dcf246d77876d00b0d79582c2cb829510430231e4472bffb4e10efee6b8ae4b"}
Aug 04 15:13:30.610731 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.610623    2569 scope.go:117] "RemoveContainer" containerID="9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"
Aug 04 15:13:30.618229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.618161    2569 scope.go:117] "RemoveContainer" containerID="b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"
Aug 04 15:13:30.625519 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.625495    2569 scope.go:117] "RemoveContainer" containerID="7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"
Aug 04 15:13:30.632833 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.632814    2569 scope.go:117] "RemoveContainer" containerID="c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"
Aug 04 15:13:30.639708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.639688    2569 scope.go:117] "RemoveContainer" containerID="96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"
Aug 04 15:13:30.647497 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.647475    2569 scope.go:117] "RemoveContainer" containerID="a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"
Aug 04 15:13:30.648759 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.648739    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:13:30.655188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.655162    2569 scope.go:117] "RemoveContainer" containerID="726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"
Aug 04 15:13:30.659366 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.659302    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:13:30.663671 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.663630    2569 scope.go:117] "RemoveContainer" containerID="9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"
Aug 04 15:13:30.663959 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.663941    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856\": container with ID starting with 9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856 not found: ID does not exist" containerID="9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"
Aug 04 15:13:30.664013 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.663971    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"} err="failed to get container status \"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856\": rpc error: code = NotFound desc = could not find container \"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856\": container with ID starting with 9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856 not found: ID does not exist"
Aug 04 15:13:30.664013 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.663991    2569 scope.go:117] "RemoveContainer" containerID="b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"
Aug 04 15:13:30.664235 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.664217    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c\": container with ID starting with b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c not found: ID does not exist" containerID="b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"
Aug 04 15:13:30.664302 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664244    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"} err="failed to get container status \"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c\": rpc error: code = NotFound desc = could not find container \"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c\": container with ID starting with b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c not found: ID does not exist"
Aug 04 15:13:30.664302 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664267    2569 scope.go:117] "RemoveContainer" containerID="7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"
Aug 04 15:13:30.664516 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.664497    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398\": container with ID starting with 7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398 not found: ID does not exist" containerID="7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"
Aug 04 15:13:30.664557 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664522    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"} err="failed to get container status \"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398\": rpc error: code = NotFound desc = could not find container \"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398\": container with ID starting with 7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398 not found: ID does not exist"
Aug 04 15:13:30.664557 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664539    2569 scope.go:117] "RemoveContainer" containerID="c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"
Aug 04 15:13:30.664791 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.664775    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5\": container with ID starting with c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5 not found: ID does not exist" containerID="c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"
Aug 04 15:13:30.664862 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664799    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"} err="failed to get container status \"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5\": rpc error: code = NotFound desc = could not find container \"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5\": container with ID starting with c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5 not found: ID does not exist"
Aug 04 15:13:30.664862 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664817    2569 scope.go:117] "RemoveContainer" containerID="96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664911    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-web-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664926    2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-cluster-tls-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664937    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664947    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664959    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnjr7\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-kube-api-access-vnjr7\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664973    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1fcf31c6-a871-4584-aff1-89cbab0b99c4-tls-assets\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.664984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664982    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-main-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664990    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcf31c6-a871-4584-aff1-89cbab0b99c4-metrics-client-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.664998    2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665006    2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1fcf31c6-a871-4584-aff1-89cbab0b99c4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665014    2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-alertmanager-main-db\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665021    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1fcf31c6-a871-4584-aff1-89cbab0b99c4-config-out\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.665085    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31\": container with ID starting with 96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31 not found: ID does not exist" containerID="96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665106    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"} err="failed to get container status \"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31\": rpc error: code = NotFound desc = could not find container \"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31\": container with ID starting with 96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31 not found: ID does not exist"
Aug 04 15:13:30.665190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665121    2569 scope.go:117] "RemoveContainer" containerID="a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"
Aug 04 15:13:30.665478 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.665349    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd\": container with ID starting with a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd not found: ID does not exist" containerID="a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"
Aug 04 15:13:30.665478 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665365    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"} err="failed to get container status \"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd\": rpc error: code = NotFound desc = could not find container \"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd\": container with ID starting with a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd not found: ID does not exist"
Aug 04 15:13:30.665478 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665378    2569 scope.go:117] "RemoveContainer" containerID="726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"
Aug 04 15:13:30.665604 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:30.665588    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea\": container with ID starting with 726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea not found: ID does not exist" containerID="726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"
Aug 04 15:13:30.665689 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665608    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"} err="failed to get container status \"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea\": rpc error: code = NotFound desc = could not find container \"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea\": container with ID starting with 726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea not found: ID does not exist"
Aug 04 15:13:30.665689 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665623    2569 scope.go:117] "RemoveContainer" containerID="9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"
Aug 04 15:13:30.665853 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665834    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856"} err="failed to get container status \"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856\": rpc error: code = NotFound desc = could not find container \"9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856\": container with ID starting with 9182c04e1b56cfe6ef5bfa541a8c955a2541a03df8fc0c7f5d4248cd902d8856 not found: ID does not exist"
Aug 04 15:13:30.665920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.665854    2569 scope.go:117] "RemoveContainer" containerID="b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"
Aug 04 15:13:30.666070 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666053    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c"} err="failed to get container status \"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c\": rpc error: code = NotFound desc = could not find container \"b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c\": container with ID starting with b832a06414a2c97caf9a14ea093d1e028788bf95b4c90c4d671e6727052ce01c not found: ID does not exist"
Aug 04 15:13:30.666131 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666072    2569 scope.go:117] "RemoveContainer" containerID="7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"
Aug 04 15:13:30.666326 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666306    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398"} err="failed to get container status \"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398\": rpc error: code = NotFound desc = could not find container \"7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398\": container with ID starting with 7a54c1950f214b611d815d73e6fdc8dcf6193b31448ff1f6a1c0876b893ac398 not found: ID does not exist"
Aug 04 15:13:30.666364 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666328    2569 scope.go:117] "RemoveContainer" containerID="c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"
Aug 04 15:13:30.666529 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666512    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5"} err="failed to get container status \"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5\": rpc error: code = NotFound desc = could not find container \"c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5\": container with ID starting with c25735eb71ec8c97541d1418109180618544564eab30b422979de639b85273e5 not found: ID does not exist"
Aug 04 15:13:30.666572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666529    2569 scope.go:117] "RemoveContainer" containerID="96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"
Aug 04 15:13:30.666750 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666733    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31"} err="failed to get container status \"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31\": rpc error: code = NotFound desc = could not find container \"96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31\": container with ID starting with 96a9f20a136229ff1b811fb72408697018216005727d09102ef59426acd6fe31 not found: ID does not exist"
Aug 04 15:13:30.666828 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666752    2569 scope.go:117] "RemoveContainer" containerID="a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"
Aug 04 15:13:30.666955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666939    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd"} err="failed to get container status \"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd\": rpc error: code = NotFound desc = could not find container \"a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd\": container with ID starting with a87f34dc226854f70b8d5d7b467d4e20df5a030f7158331c11bdd600f28250cd not found: ID does not exist"
Aug 04 15:13:30.667000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.666956    2569 scope.go:117] "RemoveContainer" containerID="726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"
Aug 04 15:13:30.667135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.667120    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea"} err="failed to get container status \"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea\": rpc error: code = NotFound desc = could not find container \"726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea\": container with ID starting with 726d963cb496a9da693f680616324e9ecfb740dacb23cf9d8158bdbe6b175bea not found: ID does not exist"
Aug 04 15:13:30.719145 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719111    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:13:30.719364 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719352    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-web"
Aug 04 15:13:30.719405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719366    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-web"
Aug 04 15:13:30.719405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719383    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="init-config-reloader"
Aug 04 15:13:30.719405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719389    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="init-config-reloader"
Aug 04 15:13:30.719405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719396    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-metric"
Aug 04 15:13:30.719405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719402    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-metric"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719410    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="alertmanager"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719416    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="alertmanager"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719424    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35546526-2227-47ec-aca2-2be57733619b" containerName="console"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719429    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="35546526-2227-47ec-aca2-2be57733619b" containerName="console"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719436    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="config-reloader"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719441    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="config-reloader"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719446    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="prom-label-proxy"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719451    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="prom-label-proxy"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719460    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719465    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719504    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="alertmanager"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719511    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719517    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-metric"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719524    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="kube-rbac-proxy-web"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719529    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="config-reloader"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719535    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="35546526-2227-47ec-aca2-2be57733619b" containerName="console"
Aug 04 15:13:30.719547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.719539    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" containerName="prom-label-proxy"
Aug 04 15:13:30.724591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.724574    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.727799 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.727770    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 04 15:13:30.727925 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.727899    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 04 15:13:30.727997 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.727905    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 04 15:13:30.728048 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.728036    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 04 15:13:30.728250 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.728232    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 04 15:13:30.728339 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.728261    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-m5lvm\""
Aug 04 15:13:30.728339 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.728309    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 04 15:13:30.728477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.728459    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 04 15:13:30.728529 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.728484    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 04 15:13:30.734495 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.734476    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 04 15:13:30.752705 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.752674    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:13:30.866483 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-web-config\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866499    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4dcbf92-c560-49b9-b93e-a64401b60761-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866522    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs62g\" (UniqueName: \"kubernetes.io/projected/b4dcbf92-c560-49b9-b93e-a64401b60761-kube-api-access-xs62g\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-config-volume\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4dcbf92-c560-49b9-b93e-a64401b60761-config-out\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dcbf92-c560-49b9-b93e-a64401b60761-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866806    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4dcbf92-c560-49b9-b93e-a64401b60761-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.866972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.866956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b4dcbf92-c560-49b9-b93e-a64401b60761-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968130 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968091    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4dcbf92-c560-49b9-b93e-a64401b60761-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968130 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968134    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968263 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968315 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968290    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b4dcbf92-c560-49b9-b93e-a64401b60761-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968385 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-web-config\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968441 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4dcbf92-c560-49b9-b93e-a64401b60761-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.968491 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs62g\" (UniqueName: \"kubernetes.io/projected/b4dcbf92-c560-49b9-b93e-a64401b60761-kube-api-access-xs62g\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-config-volume\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4dcbf92-c560-49b9-b93e-a64401b60761-config-out\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dcbf92-c560-49b9-b93e-a64401b60761-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.969226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.969175    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4dcbf92-c560-49b9-b93e-a64401b60761-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.970908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.968740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b4dcbf92-c560-49b9-b93e-a64401b60761-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.970908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.970847    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dcbf92-c560-49b9-b93e-a64401b60761-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.971296 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.971272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.971296 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.971283    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b4dcbf92-c560-49b9-b93e-a64401b60761-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.971540 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.971486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.971945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.971923    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-web-config\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.972061 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.972003    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-config-volume\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.972061 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.972002    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.972312 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.972298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.972771 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.972755    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b4dcbf92-c560-49b9-b93e-a64401b60761-config-out\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.973141 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.973126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b4dcbf92-c560-49b9-b93e-a64401b60761-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:30.988429 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:30.988398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs62g\" (UniqueName: \"kubernetes.io/projected/b4dcbf92-c560-49b9-b93e-a64401b60761-kube-api-access-xs62g\") pod \"alertmanager-main-0\" (UID: \"b4dcbf92-c560-49b9-b93e-a64401b60761\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:31.035051 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:31.035021    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 04 15:13:31.188356 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:31.188271    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 04 15:13:31.191572 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:13:31.191546    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4dcbf92_c560_49b9_b93e_a64401b60761.slice/crio-e637f9f6e192d2b77a6d9963545542e1006d183bacfb4a6a9a3f5dad3d21d1f7 WatchSource:0}: Error finding container e637f9f6e192d2b77a6d9963545542e1006d183bacfb4a6a9a3f5dad3d21d1f7: Status 404 returned error can't find the container with id e637f9f6e192d2b77a6d9963545542e1006d183bacfb4a6a9a3f5dad3d21d1f7
Aug 04 15:13:31.616225 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:31.616189    2569 generic.go:356] "Generic (PLEG): container finished" podID="b4dcbf92-c560-49b9-b93e-a64401b60761" containerID="096c48f83614236f16ed02c221435fda772955a89aee02f4ad869d7c7624c8c3" exitCode=0
Aug 04 15:13:31.616613 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:31.616232    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerDied","Data":"096c48f83614236f16ed02c221435fda772955a89aee02f4ad869d7c7624c8c3"}
Aug 04 15:13:31.616613 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:31.616285    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"e637f9f6e192d2b77a6d9963545542e1006d183bacfb4a6a9a3f5dad3d21d1f7"}
Aug 04 15:13:31.963653 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:31.963591    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fcf31c6-a871-4584-aff1-89cbab0b99c4" path="/var/lib/kubelet/pods/1fcf31c6-a871-4584-aff1-89cbab0b99c4/volumes"
Aug 04 15:13:32.622513 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.622476    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"23a20cddc14d4ebaf1cd1c7adb63d56da00d29b19e65fc4f2306d94dfb6603d7"}
Aug 04 15:13:32.622948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.622544    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"53a5e657fd66aba0887ba2a8c51c0c5f45d7e0cb79ca07fd210a32ed0bb1ee6d"}
Aug 04 15:13:32.622948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.622571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"d6e529de931f2d0f3af3e8b137fa420d2f58b9399303bbeeb6097138c0b3f2cc"}
Aug 04 15:13:32.622948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.622595    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"f5c75ec2e2b439eecf5029e6ab6b212b3ec490e2056c05c117fd22128d694ce7"}
Aug 04 15:13:32.622948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.622619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"fd6582197f9048e2b577c6d5f863945591f267df5aea032ac224b45a53f48f85"}
Aug 04 15:13:32.622948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.622657    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b4dcbf92-c560-49b9-b93e-a64401b60761","Type":"ContainerStarted","Data":"c2e861e536d4a726487169752746861eb87796889b67dc1708048f57afe786b5"}
Aug 04 15:13:32.671623 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:32.671570    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.652648757 podStartE2EDuration="2.671554798s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.906041ms" isStatefulPod=false podCreationTimestamp="2026-08-04 15:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:32.671339384 +0000 UTC m=+155.177637440" watchObservedRunningTime="2026-08-04 15:13:32.671554798 +0000 UTC m=+155.177852853"
Aug 04 15:13:33.642099 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642066    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:13:33.642506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642378    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="prometheus" containerID="cri-o://be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2" gracePeriod=600
Aug 04 15:13:33.642506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642410    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy" containerID="cri-o://8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77" gracePeriod=600
Aug 04 15:13:33.642506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642438    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-web" containerID="cri-o://c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914" gracePeriod=600
Aug 04 15:13:33.642506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642478    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="config-reloader" containerID="cri-o://b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146" gracePeriod=600
Aug 04 15:13:33.642758 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642413    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="thanos-sidecar" containerID="cri-o://93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15" gracePeriod=600
Aug 04 15:13:33.642758 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.642484    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89" gracePeriod=600
Aug 04 15:13:33.883653 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.883615    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:33.993252 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993156    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-2\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993252 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993198    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993252 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993230    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-web-config\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993525 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxjpg\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-kube-api-access-nxjpg\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993525 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993300    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-tls\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993525 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993322    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-grpc-tls\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993525 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993488    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-0\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993527    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-db\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993529    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.993708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993557    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-kubelet-serving-ca-bundle\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993585    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config-out\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-thanos-prometheus-http-client-file\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.993708 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993681    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993716    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-serving-certs-ca-bundle\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993741    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-kube-rbac-proxy\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993769    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-tls-assets\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-metrics-client-ca\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993843    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-metrics-client-certs\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993887    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-1\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-trusted-ca-bundle\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.993986    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\" (UID: \"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8\") "
Aug 04 15:13:33.994387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.994267    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:33.994809 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.994763    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.995587 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.995270    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.995587 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.995423    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.995587 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.995471    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:13:33.995841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.995776    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.995950 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.995912    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.996074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.995978    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:13:33.996747 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.996694    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.996984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.996942    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-kube-api-access-nxjpg" (OuterVolumeSpecName: "kube-api-access-nxjpg") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "kube-api-access-nxjpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:13:33.997192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.997164    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.998112 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.998072    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.998388 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.998355    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.998388 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.998370    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.998552 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.998419    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.999287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.999262    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:13:33.999384 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.999318    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config-out" (OuterVolumeSpecName: "config-out") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:13:33.999471 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.999450    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:33.999667 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:33.999618    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config" (OuterVolumeSpecName: "config") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:34.008243 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.008213    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-web-config" (OuterVolumeSpecName: "web-config") pod "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" (UID: "822c63a7-acbc-4e8b-9b01-a7ef9fea43a8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:13:34.095393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095354    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095388    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-grpc-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095403    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095415    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-db\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095428    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095440    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config-out\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095452    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095462    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095474    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095487    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-kube-rbac-proxy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095498    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-tls-assets\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095509    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-configmap-metrics-client-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095522    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-metrics-client-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095533    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095546    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095559    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095573    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095587    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-web-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.095691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.095599    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxjpg\" (UniqueName: \"kubernetes.io/projected/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8-kube-api-access-nxjpg\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:13:34.639382 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639341    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89" exitCode=0
Aug 04 15:13:34.639382 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639379    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77" exitCode=0
Aug 04 15:13:34.639382 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639389    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914" exitCode=0
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639398    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15" exitCode=0
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639407    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146" exitCode=0
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639415    2569 generic.go:356] "Generic (PLEG): container finished" podID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2" exitCode=0
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639423    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"}
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639456    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"}
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639550    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"}
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639574    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"}
Aug 04 15:13:34.639616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639595    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.639914 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639597    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"}
Aug 04 15:13:34.639914 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639680    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"}
Aug 04 15:13:34.639914 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.639704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"822c63a7-acbc-4e8b-9b01-a7ef9fea43a8","Type":"ContainerDied","Data":"d5a1ee3d5c910a2338e41df034e9445ba3402dee07864b0cded812f131b2e8c8"}
Aug 04 15:13:34.647857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.647655    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.667067 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.667048    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.670735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.670709    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:13:34.674735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.674714    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.677783 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.677757    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:13:34.682687 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.682668    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.689841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.689819    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.697511 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.697491    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.704978 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.704953    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.705313 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.705283    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.705454 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.705325    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"} err="failed to get container status \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist"
Aug 04 15:13:34.705454 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.705350    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.705759 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.705732    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.705867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.705769    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"} err="failed to get container status \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist"
Aug 04 15:13:34.705867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.705798    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.706154 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.706131    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.706206 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.706162    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"} err="failed to get container status \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist"
Aug 04 15:13:34.706206 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.706181    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.706422 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.706405    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.706466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.706424    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"} err="failed to get container status \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist"
Aug 04 15:13:34.706466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.706438    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.706700 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.706679    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.706789 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.706705    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"} err="failed to get container status \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist"
Aug 04 15:13:34.706789 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.706723    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.707009 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.706991    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.707044 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707012    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"} err="failed to get container status \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist"
Aug 04 15:13:34.707044 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707027    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.707313 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:13:34.707295    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.707366 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707319    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"} err="failed to get container status \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist"
Aug 04 15:13:34.707366 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707344    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.707570 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707553    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"} err="failed to get container status \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist"
Aug 04 15:13:34.707607 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707571    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.707829 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707811    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"} err="failed to get container status \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist"
Aug 04 15:13:34.707877 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.707829    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.708043 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708027    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"} err="failed to get container status \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist"
Aug 04 15:13:34.708084 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708051    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.708280 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708262    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"} err="failed to get container status \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist"
Aug 04 15:13:34.708320 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708281    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.708472 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708456    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"} err="failed to get container status \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist"
Aug 04 15:13:34.708524 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708472    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.708691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708673    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"} err="failed to get container status \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist"
Aug 04 15:13:34.708740 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708693    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.708884 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708868    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"} err="failed to get container status \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist"
Aug 04 15:13:34.708920 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.708885    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.709102 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709084    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"} err="failed to get container status \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist"
Aug 04 15:13:34.709150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709101    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.709298 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709280    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"} err="failed to get container status \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist"
Aug 04 15:13:34.709298 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709298    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.709524 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709492    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"} err="failed to get container status \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist"
Aug 04 15:13:34.709524 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709523    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.709778 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709762    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"} err="failed to get container status \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist"
Aug 04 15:13:34.709820 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709779    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.709997 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709967    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"} err="failed to get container status \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist"
Aug 04 15:13:34.709997 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.709988    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.710213 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710192    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"} err="failed to get container status \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist"
Aug 04 15:13:34.710256 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710214    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.710454 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710434    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"} err="failed to get container status \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist"
Aug 04 15:13:34.710515 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710455    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.710687 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710671    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"} err="failed to get container status \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist"
Aug 04 15:13:34.710726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710688    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.710913 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710896    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"} err="failed to get container status \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist"
Aug 04 15:13:34.710980 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.710915    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.711139 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711117    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"} err="failed to get container status \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist"
Aug 04 15:13:34.711193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711140    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.711340 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711322    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"} err="failed to get container status \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist"
Aug 04 15:13:34.711421 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711342    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.711577 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711557    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"} err="failed to get container status \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist"
Aug 04 15:13:34.711577 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711575    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.711799 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711774    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"} err="failed to get container status \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist"
Aug 04 15:13:34.711799 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.711797    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.712036 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712019    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"} err="failed to get container status \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist"
Aug 04 15:13:34.712079 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712037    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.712224 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712209    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"} err="failed to get container status \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist"
Aug 04 15:13:34.712262 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712224    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.712422 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712406    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"} err="failed to get container status \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist"
Aug 04 15:13:34.712466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712423    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.712628 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712612    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"} err="failed to get container status \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist"
Aug 04 15:13:34.712723 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712628    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.712879 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712863    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"} err="failed to get container status \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist"
Aug 04 15:13:34.712922 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.712879    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.713101 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713086    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"} err="failed to get container status \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist"
Aug 04 15:13:34.713136 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713102    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.713325 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713310    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"} err="failed to get container status \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist"
Aug 04 15:13:34.713362 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713325    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.713501 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713486    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"} err="failed to get container status \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist"
Aug 04 15:13:34.713542 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713501    2569 scope.go:117] "RemoveContainer" containerID="6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"
Aug 04 15:13:34.713736 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713707    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89"} err="failed to get container status \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": rpc error: code = NotFound desc = could not find container \"6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89\": container with ID starting with 6bfce01fbb96b882cd83c3de941f5efa472aaeb1d10f2faef552381d04496d89 not found: ID does not exist"
Aug 04 15:13:34.713736 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713730    2569 scope.go:117] "RemoveContainer" containerID="8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"
Aug 04 15:13:34.713944 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713928    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77"} err="failed to get container status \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": rpc error: code = NotFound desc = could not find container \"8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77\": container with ID starting with 8b8f8e0a14289daedb774cc601d42fd804c5a906c1e1d074875c693599926a77 not found: ID does not exist"
Aug 04 15:13:34.713989 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.713944    2569 scope.go:117] "RemoveContainer" containerID="c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"
Aug 04 15:13:34.714154 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714138    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914"} err="failed to get container status \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": rpc error: code = NotFound desc = could not find container \"c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914\": container with ID starting with c23b29044585687685ce6d670eee7fadb6571d944c49f84c15a2625eabc8d914 not found: ID does not exist"
Aug 04 15:13:34.714154 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714153    2569 scope.go:117] "RemoveContainer" containerID="93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"
Aug 04 15:13:34.714351 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714335    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15"} err="failed to get container status \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": rpc error: code = NotFound desc = could not find container \"93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15\": container with ID starting with 93f2291e09ad1ef135774d644a2fe049232aae177648370ed7b77ed174e06d15 not found: ID does not exist"
Aug 04 15:13:34.714389 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714352    2569 scope.go:117] "RemoveContainer" containerID="b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"
Aug 04 15:13:34.714543 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714526    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146"} err="failed to get container status \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": rpc error: code = NotFound desc = could not find container \"b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146\": container with ID starting with b530f56ba58243851f4e276b4ef864f2435b10f1757f777fe8f7f07c1f041146 not found: ID does not exist"
Aug 04 15:13:34.714579 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714544    2569 scope.go:117] "RemoveContainer" containerID="be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"
Aug 04 15:13:34.714782 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714764    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2"} err="failed to get container status \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": rpc error: code = NotFound desc = could not find container \"be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2\": container with ID starting with be7b4d8e7f1ec11be4fb5fc403a2d61c058dc3f5ef173ccb40d180dd8cadeae2 not found: ID does not exist"
Aug 04 15:13:34.714832 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714784    2569 scope.go:117] "RemoveContainer" containerID="be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"
Aug 04 15:13:34.714992 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.714976    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f"} err="failed to get container status \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": rpc error: code = NotFound desc = could not find container \"be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f\": container with ID starting with be91d5928d6fa7d0e0b57777819f5454ae153030df76831554f2d86aaef1362f not found: ID does not exist"
Aug 04 15:13:34.721537 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721512    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:13:34.721839 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721823    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-thanos"
Aug 04 15:13:34.721908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721841    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-thanos"
Aug 04 15:13:34.721908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721859    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="config-reloader"
Aug 04 15:13:34.721908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721867    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="config-reloader"
Aug 04 15:13:34.721908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721887    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy"
Aug 04 15:13:34.721908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721895    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721910    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="thanos-sidecar"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721918    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="thanos-sidecar"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721929    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-web"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721939    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-web"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721953    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="prometheus"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721961    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="prometheus"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721973    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="init-config-reloader"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.721981    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="init-config-reloader"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.722039    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="prometheus"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.722053    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="config-reloader"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.722062    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="thanos-sidecar"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.722071    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.722079    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-thanos"
Aug 04 15:13:34.722200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.722089    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" containerName="kube-rbac-proxy-web"
Aug 04 15:13:34.727313 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.727291    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.730255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.730230    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Aug 04 15:13:34.731523 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.731505    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Aug 04 15:13:34.731628 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.731545    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Aug 04 15:13:34.731713 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.731668    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Aug 04 15:13:34.731782 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.731733    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b3rl1aiq3n1cg\""
Aug 04 15:13:34.731782 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.731755    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Aug 04 15:13:34.731782 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.731761    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Aug 04 15:13:34.732408 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.732379    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Aug 04 15:13:34.732408 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.732361    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Aug 04 15:13:34.732792 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.732775    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Aug 04 15:13:34.732792 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.732786    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Aug 04 15:13:34.732945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.732835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Aug 04 15:13:34.732945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.732930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-q8ts6\""
Aug 04 15:13:34.733186 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.733167    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Aug 04 15:13:34.736028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.735985    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Aug 04 15:13:34.738224 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.738205    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Aug 04 15:13:34.745096 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.745074    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Aug 04 15:13:34.785852 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.785799    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:13:34.901857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901765    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.901857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901801    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.901857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-web-config\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzsbc\" (UniqueName: \"kubernetes.io/projected/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-kube-api-access-hzsbc\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.901971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902043    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902083    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902108    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902457 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902457 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-config\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:34.902457 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:34.902307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-config-out\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.002713 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.002670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.002713 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.002712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.002948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.002822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-config\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.002948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.002853    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-config-out\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.002948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.002880    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.002948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.002899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-web-config\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzsbc\" (UniqueName: \"kubernetes.io/projected/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-kube-api-access-hzsbc\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003193    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003226    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003773 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003773 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003773 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003538    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.003773 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.003584    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006036 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.005936    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-config-out\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006036 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-config\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006676 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-web-config\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006676 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006676 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006676 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006907 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006907 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006999 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006910    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.006999 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.006921    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.007202 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.007177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.007607 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.007580    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.008296 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.008270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.008984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.008955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.009071 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.008996    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.009135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.009084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.009340 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.009325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.014744 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.014723    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzsbc\" (UniqueName: \"kubernetes.io/projected/f17170f9-3c2d-49b6-b09f-f81fff5c8d65-kube-api-access-hzsbc\") pod \"prometheus-k8s-0\" (UID: \"f17170f9-3c2d-49b6-b09f-f81fff5c8d65\") " pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.036838 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.036797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:13:35.190728 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.190700    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Aug 04 15:13:35.193101 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:13:35.193068    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17170f9_3c2d_49b6_b09f_f81fff5c8d65.slice/crio-7e2c26114e82dd368fe1339ecf483529741214b01ffa7541e1912b59e54833ac WatchSource:0}: Error finding container 7e2c26114e82dd368fe1339ecf483529741214b01ffa7541e1912b59e54833ac: Status 404 returned error can't find the container with id 7e2c26114e82dd368fe1339ecf483529741214b01ffa7541e1912b59e54833ac
Aug 04 15:13:35.649591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.649554    2569 generic.go:356] "Generic (PLEG): container finished" podID="f17170f9-3c2d-49b6-b09f-f81fff5c8d65" containerID="63f2c291eb60233d9a69428f37d57b258dadd01336f803adf4b3bcca4f0427ff" exitCode=0
Aug 04 15:13:35.649591 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.649597    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerDied","Data":"63f2c291eb60233d9a69428f37d57b258dadd01336f803adf4b3bcca4f0427ff"}
Aug 04 15:13:35.650086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.649690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"7e2c26114e82dd368fe1339ecf483529741214b01ffa7541e1912b59e54833ac"}
Aug 04 15:13:35.964150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:35.964117    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="822c63a7-acbc-4e8b-9b01-a7ef9fea43a8" path="/var/lib/kubelet/pods/822c63a7-acbc-4e8b-9b01-a7ef9fea43a8/volumes"
Aug 04 15:13:36.657797 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.657754    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"dbe6d734fc92d0e3c8a7e11f2ae24698a1a3600ca8f6edb2061a8e45eee44331"}
Aug 04 15:13:36.658188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.657845    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"8a7def4f0ed5aa4bc349a1ef81d206a61e0541c54495a18dbb480da2058ee3af"}
Aug 04 15:13:36.658188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.657895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"f4b8ea5a8ad1fb1552cb4460b47c7e2f1af3cacc0fbae626c9750ef050c165d6"}
Aug 04 15:13:36.658188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.657938    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"2a30f087150f6938b2cf132dbc00119d8a7aeddec2f0c748512b9a005bd4592d"}
Aug 04 15:13:36.658188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.657974    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"2fea1684ab4dec354bad18ee62413befe9053d0fa3009aa515e0758fb0936a33"}
Aug 04 15:13:36.658188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.658008    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f17170f9-3c2d-49b6-b09f-f81fff5c8d65","Type":"ContainerStarted","Data":"b8106239e473f9ac3026e7fd6b2146abf8e786c62dccb434cf7e8fd681158b9b"}
Aug 04 15:13:36.699555 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:36.699488    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.68053761 podStartE2EDuration="2.699467861s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.930251ms" isStatefulPod=false podCreationTimestamp="2026-08-04 15:13:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:36.69778253 +0000 UTC m=+159.204080601" watchObservedRunningTime="2026-08-04 15:13:36.699467861 +0000 UTC m=+159.205765919"
Aug 04 15:13:40.037787 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:13:40.037727    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:14:35.037510 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:14:35.037474    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:14:35.053922 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:14:35.053892    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:14:35.891412 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:14:35.891375    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Aug 04 15:15:57.829030 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:15:57.829001    2569 kubelet.go:1648] "Image garbage collection succeeded"
Aug 04 15:16:31.124356 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.124312    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"]
Aug 04 15:16:31.127813 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.127797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.130385 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.130362    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 04 15:16:31.131221 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.131193    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 04 15:16:31.131341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.131224    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-6b8wn\""
Aug 04 15:16:31.136569 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.136542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"]
Aug 04 15:16:31.141802 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.141773    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss9dj\" (UniqueName: \"kubernetes.io/projected/4ecf8750-5198-455a-8561-20f37988019c-kube-api-access-ss9dj\") pod \"cert-manager-cainjector-559cdd8c87-zfvfp\" (UID: \"4ecf8750-5198-455a-8561-20f37988019c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.141912 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.141812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ecf8750-5198-455a-8561-20f37988019c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zfvfp\" (UID: \"4ecf8750-5198-455a-8561-20f37988019c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.242879 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.242836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss9dj\" (UniqueName: \"kubernetes.io/projected/4ecf8750-5198-455a-8561-20f37988019c-kube-api-access-ss9dj\") pod \"cert-manager-cainjector-559cdd8c87-zfvfp\" (UID: \"4ecf8750-5198-455a-8561-20f37988019c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.242879 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.242878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ecf8750-5198-455a-8561-20f37988019c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zfvfp\" (UID: \"4ecf8750-5198-455a-8561-20f37988019c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.251118 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.251083    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss9dj\" (UniqueName: \"kubernetes.io/projected/4ecf8750-5198-455a-8561-20f37988019c-kube-api-access-ss9dj\") pod \"cert-manager-cainjector-559cdd8c87-zfvfp\" (UID: \"4ecf8750-5198-455a-8561-20f37988019c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.251286 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.251121    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ecf8750-5198-455a-8561-20f37988019c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-zfvfp\" (UID: \"4ecf8750-5198-455a-8561-20f37988019c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.450132 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.450022    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"
Aug 04 15:16:31.580077 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.580043    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp"]
Aug 04 15:16:31.583239 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:16:31.583209    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ecf8750_5198_455a_8561_20f37988019c.slice/crio-74e4ceebd8b4203f0609524787386f5aa6383fde74e10b2ad7f7100e4261e0d7 WatchSource:0}: Error finding container 74e4ceebd8b4203f0609524787386f5aa6383fde74e10b2ad7f7100e4261e0d7: Status 404 returned error can't find the container with id 74e4ceebd8b4203f0609524787386f5aa6383fde74e10b2ad7f7100e4261e0d7
Aug 04 15:16:31.585156 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:31.585138    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 15:16:32.303219 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:32.303176    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp" event={"ID":"4ecf8750-5198-455a-8561-20f37988019c","Type":"ContainerStarted","Data":"74e4ceebd8b4203f0609524787386f5aa6383fde74e10b2ad7f7100e4261e0d7"}
Aug 04 15:16:35.317549 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:35.317515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp" event={"ID":"4ecf8750-5198-455a-8561-20f37988019c","Type":"ContainerStarted","Data":"90afad26f6578ea3e2707194e01a4e32e46f741ba8ec1de1d54cb2eadbd6f70a"}
Aug 04 15:16:35.333519 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:16:35.333466    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-zfvfp" podStartSLOduration=1.084946306 podStartE2EDuration="4.333446997s" totalImagesPullingTime="3.248500691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:16:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:16:35.332552875 +0000 UTC m=+337.838850931" watchObservedRunningTime="2026-08-04 15:16:35.333446997 +0000 UTC m=+337.839745053"
Aug 04 15:17:04.243255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.243167    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"]
Aug 04 15:17:04.246567 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.246548    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.250787 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.250756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 04 15:17:04.250787 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.250775    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 04 15:17:04.250787 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.250756    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 04 15:17:04.251032 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.250756    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 04 15:17:04.251032 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.250810    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 04 15:17:04.251547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.251528    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jm652\""
Aug 04 15:17:04.260193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.260160    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"]
Aug 04 15:17:04.322596 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.322551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-metrics-cert\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.322808 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.322608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x9c7\" (UniqueName: \"kubernetes.io/projected/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-kube-api-access-4x9c7\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.322808 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.322633    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-cert\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.322808 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.322752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-manager-config\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.423918 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.423881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-metrics-cert\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.423918 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.423920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9c7\" (UniqueName: \"kubernetes.io/projected/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-kube-api-access-4x9c7\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.424179 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.423940    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-cert\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.424179 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.423973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-manager-config\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.424610 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.424587    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-manager-config\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.426399 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.426367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-metrics-cert\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.426517 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.426470    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-cert\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.435541 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.435512    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9c7\" (UniqueName: \"kubernetes.io/projected/74b5ab48-00f7-4391-830c-f3e3bc5fbddc-kube-api-access-4x9c7\") pod \"lws-controller-manager-59d7b7db79-wh7nn\" (UID: \"74b5ab48-00f7-4391-830c-f3e3bc5fbddc\") " pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.556544 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.556443    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:04.704789 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:04.704745    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"]
Aug 04 15:17:04.708365 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:17:04.708337    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b5ab48_00f7_4391_830c_f3e3bc5fbddc.slice/crio-52f6603d93e375534d33a3c7e65208e32d884cabaf829c25a4aba082b0347566 WatchSource:0}: Error finding container 52f6603d93e375534d33a3c7e65208e32d884cabaf829c25a4aba082b0347566: Status 404 returned error can't find the container with id 52f6603d93e375534d33a3c7e65208e32d884cabaf829c25a4aba082b0347566
Aug 04 15:17:05.425698 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.425660    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn" event={"ID":"74b5ab48-00f7-4391-830c-f3e3bc5fbddc","Type":"ContainerStarted","Data":"52f6603d93e375534d33a3c7e65208e32d884cabaf829c25a4aba082b0347566"}
Aug 04 15:17:05.604598 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.604558    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"]
Aug 04 15:17:05.609316 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.609291    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.612075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.612025    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Aug 04 15:17:05.612075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.612060    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Aug 04 15:17:05.612075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.612042    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Aug 04 15:17:05.612331 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.612046    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fjn9c\""
Aug 04 15:17:05.612331 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.612048    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Aug 04 15:17:05.626912 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.626881    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"]
Aug 04 15:17:05.737841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.737430    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99821cb7-a349-4786-8d61-9437543e6269-webhook-cert\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.737841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.737489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99821cb7-a349-4786-8d61-9437543e6269-manifests\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.737841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.737567    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99821cb7-a349-4786-8d61-9437543e6269-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.737841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.737592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99821cb7-a349-4786-8d61-9437543e6269-tmp\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.737841 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.737663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94d26\" (UniqueName: \"kubernetes.io/projected/99821cb7-a349-4786-8d61-9437543e6269-kube-api-access-94d26\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.838847 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.838802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99821cb7-a349-4786-8d61-9437543e6269-webhook-cert\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.839048 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.838858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99821cb7-a349-4786-8d61-9437543e6269-manifests\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.839048 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.838938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99821cb7-a349-4786-8d61-9437543e6269-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.839048 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.838967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99821cb7-a349-4786-8d61-9437543e6269-tmp\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.839048 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.839021    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94d26\" (UniqueName: \"kubernetes.io/projected/99821cb7-a349-4786-8d61-9437543e6269-kube-api-access-94d26\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.839413 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.839380    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/99821cb7-a349-4786-8d61-9437543e6269-manifests\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.842094 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.842065    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/99821cb7-a349-4786-8d61-9437543e6269-webhook-cert\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.842240 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.842182    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99821cb7-a349-4786-8d61-9437543e6269-tmp\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.842406 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.842385    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/99821cb7-a349-4786-8d61-9437543e6269-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.853511 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.853468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94d26\" (UniqueName: \"kubernetes.io/projected/99821cb7-a349-4786-8d61-9437543e6269-kube-api-access-94d26\") pod \"opendatahub-operator-controller-manager-77c9c696b6-5xgqz\" (UID: \"99821cb7-a349-4786-8d61-9437543e6269\") " pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:05.919924 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:05.919875    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:06.078673 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:06.078622    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"]
Aug 04 15:17:06.080489 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:17:06.080462    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99821cb7_a349_4786_8d61_9437543e6269.slice/crio-223deeda2cbe247c5c9d9f9386d5848effd0e10bde2aa7df965730e932247d6d WatchSource:0}: Error finding container 223deeda2cbe247c5c9d9f9386d5848effd0e10bde2aa7df965730e932247d6d: Status 404 returned error can't find the container with id 223deeda2cbe247c5c9d9f9386d5848effd0e10bde2aa7df965730e932247d6d
Aug 04 15:17:06.432044 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:06.431991    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz" event={"ID":"99821cb7-a349-4786-8d61-9437543e6269","Type":"ContainerStarted","Data":"223deeda2cbe247c5c9d9f9386d5848effd0e10bde2aa7df965730e932247d6d"}
Aug 04 15:17:08.443210 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:08.443170    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn" event={"ID":"74b5ab48-00f7-4391-830c-f3e3bc5fbddc","Type":"ContainerStarted","Data":"cca93f0acd0eefbfab54d6ceb95bfd58800b052d560656777c7ebf4c2a27aed3"}
Aug 04 15:17:08.443703 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:08.443300    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:08.463033 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:08.462969    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn" podStartSLOduration=1.619492217 podStartE2EDuration="4.462947994s" totalImagesPullingTime="2.843455777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:17:08.462443296 +0000 UTC m=+370.968741353" watchObservedRunningTime="2026-08-04 15:17:08.462947994 +0000 UTC m=+370.969246051"
Aug 04 15:17:10.452655 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:10.452608    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz" event={"ID":"99821cb7-a349-4786-8d61-9437543e6269","Type":"ContainerStarted","Data":"0c30303fb95106c0f2aedb51b8bf9d450a1e0178bc87b3afd9083f4535579ca2"}
Aug 04 15:17:11.458028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:11.457990    2569 generic.go:356] "Generic (PLEG): container finished" podID="99821cb7-a349-4786-8d61-9437543e6269" containerID="0c30303fb95106c0f2aedb51b8bf9d450a1e0178bc87b3afd9083f4535579ca2" exitCode=0
Aug 04 15:17:11.458534 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:11.458074    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz" event={"ID":"99821cb7-a349-4786-8d61-9437543e6269","Type":"ContainerDied","Data":"0c30303fb95106c0f2aedb51b8bf9d450a1e0178bc87b3afd9083f4535579ca2"}
Aug 04 15:17:12.465730 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:12.465685    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz" event={"ID":"99821cb7-a349-4786-8d61-9437543e6269","Type":"ContainerStarted","Data":"a25675a7d3afa8f4bd931c104b65507ce9fbb2f5b86d4dc29345ef0f3b94134f"}
Aug 04 15:17:12.466252 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:12.465860    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:12.490864 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:12.490812    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz" podStartSLOduration=2.017628587 podStartE2EDuration="7.490794223s" totalImagesPullingTime="4.394853873s" totalInitContainerRuntime="1.078311763s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:17:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:17:12.489272984 +0000 UTC m=+374.995571050" watchObservedRunningTime="2026-08-04 15:17:12.490794223 +0000 UTC m=+374.997092279"
Aug 04 15:17:19.449669 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:19.449623    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59d7b7db79-wh7nn"
Aug 04 15:17:23.472134 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:23.472102    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77c9c696b6-5xgqz"
Aug 04 15:17:28.952948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.952907    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"]
Aug 04 15:17:28.956863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.956845    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:28.970287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.970262    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 04 15:17:28.970287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.970262    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Aug 04 15:17:28.970986 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.970969    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 04 15:17:28.971209 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.971192    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Aug 04 15:17:28.971645 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.971621    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-29mzv\""
Aug 04 15:17:28.977003 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:28.976979    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"]
Aug 04 15:17:29.035126 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.035082    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72j7n\" (UniqueName: \"kubernetes.io/projected/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-kube-api-access-72j7n\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.035126 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.035131    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-tls-certs\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.035350 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.035178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-tmp\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.136369 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.136324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-tls-certs\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.136550 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.136387    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-tmp\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.136550 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.136433    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72j7n\" (UniqueName: \"kubernetes.io/projected/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-kube-api-access-72j7n\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.138873 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.138842    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-tmp\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.139027 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.139012    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-tls-certs\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.150229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.150200    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72j7n\" (UniqueName: \"kubernetes.io/projected/bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7-kube-api-access-72j7n\") pod \"kube-auth-proxy-86b9dff769-xlmdx\" (UID: \"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7\") " pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.267065 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.266970    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"
Aug 04 15:17:29.461259 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.461216    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx"]
Aug 04 15:17:29.531279 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:29.531187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx" event={"ID":"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7","Type":"ContainerStarted","Data":"555bf37baff0cff2993e9e960142d2f260cade21d1bb7a85142bae55a779442a"}
Aug 04 15:17:33.553330 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:33.553291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx" event={"ID":"bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7","Type":"ContainerStarted","Data":"ca53819774a2ce4927351933e0de5b7e86c335df15a86208d5b1c1968724d966"}
Aug 04 15:17:33.575076 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:33.575017    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86b9dff769-xlmdx" podStartSLOduration=2.408389222 podStartE2EDuration="5.575001024s" totalImagesPullingTime="3.166611802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:17:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:17:33.5736906 +0000 UTC m=+396.079988658" watchObservedRunningTime="2026-08-04 15:17:33.575001024 +0000 UTC m=+396.081299119"
Aug 04 15:17:40.696541 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.696505    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"]
Aug 04 15:17:40.748265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.748233    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"]
Aug 04 15:17:40.748437 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.748349    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:40.751274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.751246    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6tqtp\""
Aug 04 15:17:40.751443 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.751298    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Aug 04 15:17:40.844670 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.844560    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcjkk\" (UniqueName: \"kubernetes.io/projected/f86e84a8-cf58-42e0-8467-ebcf739b4750-kube-api-access-zcjkk\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:40.844896 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.844718    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f86e84a8-cf58-42e0-8467-ebcf739b4750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:40.945215 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.945178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcjkk\" (UniqueName: \"kubernetes.io/projected/f86e84a8-cf58-42e0-8467-ebcf739b4750-kube-api-access-zcjkk\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:40.945395 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.945267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f86e84a8-cf58-42e0-8467-ebcf739b4750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:40.945395 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:17:40.945388    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Aug 04 15:17:40.945489 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:17:40.945460    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f86e84a8-cf58-42e0-8467-ebcf739b4750-cert podName:f86e84a8-cf58-42e0-8467-ebcf739b4750 nodeName:}" failed. No retries permitted until 2026-08-04 15:17:41.445436348 +0000 UTC m=+403.951734387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f86e84a8-cf58-42e0-8467-ebcf739b4750-cert") pod "llmisvc-controller-manager-777c6bd7d5-7gpd8" (UID: "f86e84a8-cf58-42e0-8467-ebcf739b4750") : secret "llmisvc-webhook-server-cert" not found
Aug 04 15:17:40.961150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:40.961078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcjkk\" (UniqueName: \"kubernetes.io/projected/f86e84a8-cf58-42e0-8467-ebcf739b4750-kube-api-access-zcjkk\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:41.449187 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:41.449146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f86e84a8-cf58-42e0-8467-ebcf739b4750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:41.451939 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:41.451904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f86e84a8-cf58-42e0-8467-ebcf739b4750-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7gpd8\" (UID: \"f86e84a8-cf58-42e0-8467-ebcf739b4750\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:41.658025 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:41.657993    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:41.824578 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:41.824546    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"]
Aug 04 15:17:41.829246 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:17:41.829215    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf86e84a8_cf58_42e0_8467_ebcf739b4750.slice/crio-d8271e01a1bc151caca626d0c523eec09da8dd801e2f970483ebe7b0c77e1ce8 WatchSource:0}: Error finding container d8271e01a1bc151caca626d0c523eec09da8dd801e2f970483ebe7b0c77e1ce8: Status 404 returned error can't find the container with id d8271e01a1bc151caca626d0c523eec09da8dd801e2f970483ebe7b0c77e1ce8
Aug 04 15:17:42.589379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:42.589344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8" event={"ID":"f86e84a8-cf58-42e0-8467-ebcf739b4750","Type":"ContainerStarted","Data":"d8271e01a1bc151caca626d0c523eec09da8dd801e2f970483ebe7b0c77e1ce8"}
Aug 04 15:17:45.602448 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:45.602407    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8" event={"ID":"f86e84a8-cf58-42e0-8467-ebcf739b4750","Type":"ContainerStarted","Data":"f79296f292feb8c9f6822ad1f287234059c64e4680878d3104376a2eb472583e"}
Aug 04 15:17:45.602876 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:45.602545    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:17:45.620794 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:17:45.620715    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8" podStartSLOduration=2.5374210870000002 podStartE2EDuration="5.620698936s" totalImagesPullingTime="3.083277849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:17:45.620206338 +0000 UTC m=+408.126504398" watchObservedRunningTime="2026-08-04 15:17:45.620698936 +0000 UTC m=+408.126996991"
Aug 04 15:18:16.608753 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:18:16.608720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7gpd8"
Aug 04 15:19:12.107131 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.107086    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"]
Aug 04 15:19:12.110624 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.110601    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.113264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.113234    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 04 15:19:12.113264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.113247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Aug 04 15:19:12.113484 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.113283    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Aug 04 15:19:12.113977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.113961    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 04 15:19:12.114315 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.114299    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h67gw\""
Aug 04 15:19:12.124412 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.124385    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"]
Aug 04 15:19:12.126978 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.126950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/765f70f7-988d-464f-a86a-28e7bd35b0cd-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.127115 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.126992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/765f70f7-988d-464f-a86a-28e7bd35b0cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.127115 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.127065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzpdr\" (UniqueName: \"kubernetes.io/projected/765f70f7-988d-464f-a86a-28e7bd35b0cd-kube-api-access-gzpdr\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.227665 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.227597    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/765f70f7-988d-464f-a86a-28e7bd35b0cd-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.227665 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.227665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/765f70f7-988d-464f-a86a-28e7bd35b0cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.227977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.227720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzpdr\" (UniqueName: \"kubernetes.io/projected/765f70f7-988d-464f-a86a-28e7bd35b0cd-kube-api-access-gzpdr\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.227977 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:19:12.227832    2569 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Aug 04 15:19:12.227977 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:19:12.227924    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/765f70f7-988d-464f-a86a-28e7bd35b0cd-plugin-serving-cert podName:765f70f7-988d-464f-a86a-28e7bd35b0cd nodeName:}" failed. No retries permitted until 2026-08-04 15:19:12.727899512 +0000 UTC m=+495.234197548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/765f70f7-988d-464f-a86a-28e7bd35b0cd-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-fwn7r" (UID: "765f70f7-988d-464f-a86a-28e7bd35b0cd") : secret "plugin-serving-cert" not found
Aug 04 15:19:12.228297 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.228276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/765f70f7-988d-464f-a86a-28e7bd35b0cd-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.241168 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.241142    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzpdr\" (UniqueName: \"kubernetes.io/projected/765f70f7-988d-464f-a86a-28e7bd35b0cd-kube-api-access-gzpdr\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.730777 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.730728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/765f70f7-988d-464f-a86a-28e7bd35b0cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:12.733219 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:12.733189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/765f70f7-988d-464f-a86a-28e7bd35b0cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-fwn7r\" (UID: \"765f70f7-988d-464f-a86a-28e7bd35b0cd\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:13.019657 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:13.019542    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"
Aug 04 15:19:13.160392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:13.160305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r"]
Aug 04 15:19:13.162666 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:19:13.162622    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod765f70f7_988d_464f_a86a_28e7bd35b0cd.slice/crio-2c8984bc2984a17fbd8cb8df3b3bc2d5d48a030766cfe6305e0d31637a82db97 WatchSource:0}: Error finding container 2c8984bc2984a17fbd8cb8df3b3bc2d5d48a030766cfe6305e0d31637a82db97: Status 404 returned error can't find the container with id 2c8984bc2984a17fbd8cb8df3b3bc2d5d48a030766cfe6305e0d31637a82db97
Aug 04 15:19:13.936444 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:13.936408    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r" event={"ID":"765f70f7-988d-464f-a86a-28e7bd35b0cd","Type":"ContainerStarted","Data":"2c8984bc2984a17fbd8cb8df3b3bc2d5d48a030766cfe6305e0d31637a82db97"}
Aug 04 15:19:19.967243 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:19.967203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r" event={"ID":"765f70f7-988d-464f-a86a-28e7bd35b0cd","Type":"ContainerStarted","Data":"f511464741817aa04f86442a1d9dee939a4fe7ebfa7ac35b68661138c43ba747"}
Aug 04 15:19:19.984615 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:19.984557    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-fwn7r" podStartSLOduration=1.9267983690000001 podStartE2EDuration="7.984540616s" totalImagesPullingTime="6.057742247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:19:19.983955189 +0000 UTC m=+502.490253246" watchObservedRunningTime="2026-08-04 15:19:19.984540616 +0000 UTC m=+502.490838671"
Aug 04 15:19:53.657444 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.657347    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:19:53.660913 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.660888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.663514 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.663487    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 04 15:19:53.671945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.671920    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:19:53.685337 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.685305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:19:53.796416 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.796372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b420a609-7113-4bcc-8a37-44f8ac15c8bd-config-file\") pod \"limitador-limitador-85d9db4d5d-l7x4t\" (UID: \"b420a609-7113-4bcc-8a37-44f8ac15c8bd\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.796416 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.796420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdf2r\" (UniqueName: \"kubernetes.io/projected/b420a609-7113-4bcc-8a37-44f8ac15c8bd-kube-api-access-cdf2r\") pod \"limitador-limitador-85d9db4d5d-l7x4t\" (UID: \"b420a609-7113-4bcc-8a37-44f8ac15c8bd\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.898045 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.898005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b420a609-7113-4bcc-8a37-44f8ac15c8bd-config-file\") pod \"limitador-limitador-85d9db4d5d-l7x4t\" (UID: \"b420a609-7113-4bcc-8a37-44f8ac15c8bd\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.898220 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.898051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdf2r\" (UniqueName: \"kubernetes.io/projected/b420a609-7113-4bcc-8a37-44f8ac15c8bd-kube-api-access-cdf2r\") pod \"limitador-limitador-85d9db4d5d-l7x4t\" (UID: \"b420a609-7113-4bcc-8a37-44f8ac15c8bd\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.898685 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.898666    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b420a609-7113-4bcc-8a37-44f8ac15c8bd-config-file\") pod \"limitador-limitador-85d9db4d5d-l7x4t\" (UID: \"b420a609-7113-4bcc-8a37-44f8ac15c8bd\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.906915 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.906883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdf2r\" (UniqueName: \"kubernetes.io/projected/b420a609-7113-4bcc-8a37-44f8ac15c8bd-kube-api-access-cdf2r\") pod \"limitador-limitador-85d9db4d5d-l7x4t\" (UID: \"b420a609-7113-4bcc-8a37-44f8ac15c8bd\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:53.971899 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:53.971816    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:54.107073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.107030    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:19:54.111193 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:19:54.111154    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb420a609_7113_4bcc_8a37_44f8ac15c8bd.slice/crio-8cffca4dd46c70d00adb0dee9eda62bb4dd1a08734c54cbf38c42bcd98427f75 WatchSource:0}: Error finding container 8cffca4dd46c70d00adb0dee9eda62bb4dd1a08734c54cbf38c42bcd98427f75: Status 404 returned error can't find the container with id 8cffca4dd46c70d00adb0dee9eda62bb4dd1a08734c54cbf38c42bcd98427f75
Aug 04 15:19:54.278307 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.278218    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-z82xz"]
Aug 04 15:19:54.282559 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.282533    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:19:54.284908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.284889    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wm752\""
Aug 04 15:19:54.291719 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.291696    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-z82xz"]
Aug 04 15:19:54.409836 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.409800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgtn2\" (UniqueName: \"kubernetes.io/projected/cb781e41-3008-475e-9dae-b2ece5bb472a-kube-api-access-mgtn2\") pod \"authorino-c548b5d7d-z82xz\" (UID: \"cb781e41-3008-475e-9dae-b2ece5bb472a\") " pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:19:54.510547 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.510502    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgtn2\" (UniqueName: \"kubernetes.io/projected/cb781e41-3008-475e-9dae-b2ece5bb472a-kube-api-access-mgtn2\") pod \"authorino-c548b5d7d-z82xz\" (UID: \"cb781e41-3008-475e-9dae-b2ece5bb472a\") " pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:19:54.521737 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.521706    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgtn2\" (UniqueName: \"kubernetes.io/projected/cb781e41-3008-475e-9dae-b2ece5bb472a-kube-api-access-mgtn2\") pod \"authorino-c548b5d7d-z82xz\" (UID: \"cb781e41-3008-475e-9dae-b2ece5bb472a\") " pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:19:54.592245 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.592151    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:19:54.722587 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:54.722553    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-z82xz"]
Aug 04 15:19:54.726405 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:19:54.726367    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb781e41_3008_475e_9dae_b2ece5bb472a.slice/crio-8b5df3647d8759c4e5f33625512732159e9c3d0132d87beb1d70988ceffea330 WatchSource:0}: Error finding container 8b5df3647d8759c4e5f33625512732159e9c3d0132d87beb1d70988ceffea330: Status 404 returned error can't find the container with id 8b5df3647d8759c4e5f33625512732159e9c3d0132d87beb1d70988ceffea330
Aug 04 15:19:55.117863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:55.117825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-z82xz" event={"ID":"cb781e41-3008-475e-9dae-b2ece5bb472a","Type":"ContainerStarted","Data":"8b5df3647d8759c4e5f33625512732159e9c3d0132d87beb1d70988ceffea330"}
Aug 04 15:19:55.119342 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:55.119303    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t" event={"ID":"b420a609-7113-4bcc-8a37-44f8ac15c8bd","Type":"ContainerStarted","Data":"8cffca4dd46c70d00adb0dee9eda62bb4dd1a08734c54cbf38c42bcd98427f75"}
Aug 04 15:19:57.130567 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:57.130526    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-z82xz" event={"ID":"cb781e41-3008-475e-9dae-b2ece5bb472a","Type":"ContainerStarted","Data":"1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50"}
Aug 04 15:19:57.132133 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:57.132107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t" event={"ID":"b420a609-7113-4bcc-8a37-44f8ac15c8bd","Type":"ContainerStarted","Data":"e7b57f72d227e6cb0c27ccc9f3784f4147327154d1206c1c83d2630f22e5e588"}
Aug 04 15:19:57.132265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:57.132175    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:19:57.146916 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:57.146867    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-z82xz" podStartSLOduration=0.911180837 podStartE2EDuration="3.146847898s" totalImagesPullingTime="2.235667061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:19:57.14588074 +0000 UTC m=+539.652178789" watchObservedRunningTime="2026-08-04 15:19:57.146847898 +0000 UTC m=+539.653145932"
Aug 04 15:19:57.167477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:19:57.167422    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t" podStartSLOduration=1.322401362 podStartE2EDuration="4.167405043s" totalImagesPullingTime="2.845003681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:19:57.165712798 +0000 UTC m=+539.672010855" watchObservedRunningTime="2026-08-04 15:19:57.167405043 +0000 UTC m=+539.673703098"
Aug 04 15:20:08.137598 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:08.137552    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"
Aug 04 15:20:22.862406 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:22.862368    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f67848fb-5fhtq"]
Aug 04 15:20:22.865805 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:22.865782    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:22.868275 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:22.868248    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Aug 04 15:20:22.874259 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:22.874230    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f67848fb-5fhtq"]
Aug 04 15:20:22.968057 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:22.968016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g94sm\" (UniqueName: \"kubernetes.io/projected/d99bca71-914e-4480-84be-de2e808ae294-kube-api-access-g94sm\") pod \"authorino-6f67848fb-5fhtq\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") " pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:22.968057 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:22.968067    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d99bca71-914e-4480-84be-de2e808ae294-tls-cert\") pod \"authorino-6f67848fb-5fhtq\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") " pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:23.068945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:23.068907    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g94sm\" (UniqueName: \"kubernetes.io/projected/d99bca71-914e-4480-84be-de2e808ae294-kube-api-access-g94sm\") pod \"authorino-6f67848fb-5fhtq\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") " pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:23.069145 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:23.068967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d99bca71-914e-4480-84be-de2e808ae294-tls-cert\") pod \"authorino-6f67848fb-5fhtq\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") " pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:23.071477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:23.071450    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d99bca71-914e-4480-84be-de2e808ae294-tls-cert\") pod \"authorino-6f67848fb-5fhtq\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") " pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:23.077695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:23.077668    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g94sm\" (UniqueName: \"kubernetes.io/projected/d99bca71-914e-4480-84be-de2e808ae294-kube-api-access-g94sm\") pod \"authorino-6f67848fb-5fhtq\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") " pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:23.175760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:23.175722    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:20:23.328331 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:23.328296    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f67848fb-5fhtq"]
Aug 04 15:20:23.331451 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:20:23.331414    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd99bca71_914e_4480_84be_de2e808ae294.slice/crio-2a9df5153b03bd62bb5b7053fba822c39742c8278589ea705977a69bfbbdd3e8 WatchSource:0}: Error finding container 2a9df5153b03bd62bb5b7053fba822c39742c8278589ea705977a69bfbbdd3e8: Status 404 returned error can't find the container with id 2a9df5153b03bd62bb5b7053fba822c39742c8278589ea705977a69bfbbdd3e8
Aug 04 15:20:24.248396 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.248300    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f67848fb-5fhtq" event={"ID":"d99bca71-914e-4480-84be-de2e808ae294","Type":"ContainerStarted","Data":"732beee85065aef5295199cd5a962bd39d6902013f6bfe438da537ac030a8a0d"}
Aug 04 15:20:24.248396 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.248345    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f67848fb-5fhtq" event={"ID":"d99bca71-914e-4480-84be-de2e808ae294","Type":"ContainerStarted","Data":"2a9df5153b03bd62bb5b7053fba822c39742c8278589ea705977a69bfbbdd3e8"}
Aug 04 15:20:24.265338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.265285    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f67848fb-5fhtq" podStartSLOduration=1.609825899 podStartE2EDuration="2.265266505s" totalImagesPullingTime="655.440606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:20:24.264011513 +0000 UTC m=+566.770309582" watchObservedRunningTime="2026-08-04 15:20:24.265266505 +0000 UTC m=+566.771564562"
Aug 04 15:20:24.291548 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.291511    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-z82xz"]
Aug 04 15:20:24.291762 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.291739    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-z82xz" podUID="cb781e41-3008-475e-9dae-b2ece5bb472a" containerName="authorino" containerID="cri-o://1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50" gracePeriod=30
Aug 04 15:20:24.538976 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.538947    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:20:24.684150 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.684107    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgtn2\" (UniqueName: \"kubernetes.io/projected/cb781e41-3008-475e-9dae-b2ece5bb472a-kube-api-access-mgtn2\") pod \"cb781e41-3008-475e-9dae-b2ece5bb472a\" (UID: \"cb781e41-3008-475e-9dae-b2ece5bb472a\") "
Aug 04 15:20:24.686384 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.686358    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb781e41-3008-475e-9dae-b2ece5bb472a-kube-api-access-mgtn2" (OuterVolumeSpecName: "kube-api-access-mgtn2") pod "cb781e41-3008-475e-9dae-b2ece5bb472a" (UID: "cb781e41-3008-475e-9dae-b2ece5bb472a"). InnerVolumeSpecName "kube-api-access-mgtn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:20:24.784957 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:24.784871    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgtn2\" (UniqueName: \"kubernetes.io/projected/cb781e41-3008-475e-9dae-b2ece5bb472a-kube-api-access-mgtn2\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:20:25.253741 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.253702    2569 generic.go:356] "Generic (PLEG): container finished" podID="cb781e41-3008-475e-9dae-b2ece5bb472a" containerID="1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50" exitCode=0
Aug 04 15:20:25.254228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.253752    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-z82xz"
Aug 04 15:20:25.254228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.253793    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-z82xz" event={"ID":"cb781e41-3008-475e-9dae-b2ece5bb472a","Type":"ContainerDied","Data":"1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50"}
Aug 04 15:20:25.254228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.253851    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-z82xz" event={"ID":"cb781e41-3008-475e-9dae-b2ece5bb472a","Type":"ContainerDied","Data":"8b5df3647d8759c4e5f33625512732159e9c3d0132d87beb1d70988ceffea330"}
Aug 04 15:20:25.254228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.253875    2569 scope.go:117] "RemoveContainer" containerID="1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50"
Aug 04 15:20:25.262715 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.262695    2569 scope.go:117] "RemoveContainer" containerID="1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50"
Aug 04 15:20:25.263060 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:20:25.263037    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50\": container with ID starting with 1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50 not found: ID does not exist" containerID="1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50"
Aug 04 15:20:25.263141 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.263072    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50"} err="failed to get container status \"1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50\": rpc error: code = NotFound desc = could not find container \"1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50\": container with ID starting with 1ca7f258039fdbff4cf08811d240d4f4ef87cbc1686e3eb073981f736b0ccf50 not found: ID does not exist"
Aug 04 15:20:25.284861 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.284829    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-z82xz"]
Aug 04 15:20:25.287407 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.287376    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-z82xz"]
Aug 04 15:20:25.965053 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:20:25.965017    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb781e41-3008-475e-9dae-b2ece5bb472a" path="/var/lib/kubelet/pods/cb781e41-3008-475e-9dae-b2ece5bb472a/volumes"
Aug 04 15:21:06.199346 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:21:06.199305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:21:26.515903 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:21:26.515816    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:21:33.520159 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:21:33.520108    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:21:54.818493 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:21:54.818451    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:22:14.014836 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:14.014792    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:22:33.810963 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:33.810927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:22:57.188403 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.188301    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"]
Aug 04 15:22:57.188993 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.188853    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb781e41-3008-475e-9dae-b2ece5bb472a" containerName="authorino"
Aug 04 15:22:57.188993 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.188872    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb781e41-3008-475e-9dae-b2ece5bb472a" containerName="authorino"
Aug 04 15:22:57.189111 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.189000    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb781e41-3008-475e-9dae-b2ece5bb472a" containerName="authorino"
Aug 04 15:22:57.192071 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.192047    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.195117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.195095    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Aug 04 15:22:57.196244 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.196215    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n5m5q\""
Aug 04 15:22:57.196391 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.196246    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Aug 04 15:22:57.196391 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.196219    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Aug 04 15:22:57.204171 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.204146    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"]
Aug 04 15:22:57.351539 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.351500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.351816 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.351564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.351816 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.351607    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.351816 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.351670    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52ztr\" (UniqueName: \"kubernetes.io/projected/db23bef1-ac30-4f71-b06d-08346751fed8-kube-api-access-52ztr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.351816 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.351701    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.352001 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.351832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db23bef1-ac30-4f71-b06d-08346751fed8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.452732 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.452621    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.452732 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.452697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.452732 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.452720    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.452970 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.452746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52ztr\" (UniqueName: \"kubernetes.io/projected/db23bef1-ac30-4f71-b06d-08346751fed8-kube-api-access-52ztr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.452970 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.452765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.452970 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.452816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db23bef1-ac30-4f71-b06d-08346751fed8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.453182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.453155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.453246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.453179    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.453246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.453200    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.454947 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.454922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db23bef1-ac30-4f71-b06d-08346751fed8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.455300 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.455282    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db23bef1-ac30-4f71-b06d-08346751fed8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.462146 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.462108    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52ztr\" (UniqueName: \"kubernetes.io/projected/db23bef1-ac30-4f71-b06d-08346751fed8-kube-api-access-52ztr\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pssvn\" (UID: \"db23bef1-ac30-4f71-b06d-08346751fed8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.502135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.502095    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:22:57.642145 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.642120    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"]
Aug 04 15:22:57.645144 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:22:57.645101    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb23bef1_ac30_4f71_b06d_08346751fed8.slice/crio-d8962da217db24180ebcf597eb715965725031293190b24d7c10af9c87746d01 WatchSource:0}: Error finding container d8962da217db24180ebcf597eb715965725031293190b24d7c10af9c87746d01: Status 404 returned error can't find the container with id d8962da217db24180ebcf597eb715965725031293190b24d7c10af9c87746d01
Aug 04 15:22:57.646516 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.646494    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 15:22:57.912827 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:57.912789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn" event={"ID":"db23bef1-ac30-4f71-b06d-08346751fed8","Type":"ContainerStarted","Data":"d8962da217db24180ebcf597eb715965725031293190b24d7c10af9c87746d01"}
Aug 04 15:22:59.716175 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:59.716137    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:22:59.923220 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:59.923188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn" event={"ID":"db23bef1-ac30-4f71-b06d-08346751fed8","Type":"ContainerStarted","Data":"838767bfcdd86f70e9adbc289f36d151bdf4fb7a021c113b1a2207bed8d482c8"}
Aug 04 15:22:59.940260 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:22:59.940197    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn" podStartSLOduration=1.4596962580000001 podStartE2EDuration="2.940176326s" totalImagesPullingTime="1.480480068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:22:59.940069358 +0000 UTC m=+722.446367415" watchObservedRunningTime="2026-08-04 15:22:59.940176326 +0000 UTC m=+722.446474383"
Aug 04 15:23:07.502651 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:23:07.502603    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:23:07.503259 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:23:07.502696    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:23:07.515345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:23:07.515308    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:23:07.972246 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:23:07.972217    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pssvn"
Aug 04 15:24:07.105337 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.105301    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f67848fb-5fhtq"]
Aug 04 15:24:07.105857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.105489    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6f67848fb-5fhtq" podUID="d99bca71-914e-4480-84be-de2e808ae294" containerName="authorino" containerID="cri-o://732beee85065aef5295199cd5a962bd39d6902013f6bfe438da537ac030a8a0d" gracePeriod=30
Aug 04 15:24:07.223848 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.223817    2569 generic.go:356] "Generic (PLEG): container finished" podID="d99bca71-914e-4480-84be-de2e808ae294" containerID="732beee85065aef5295199cd5a962bd39d6902013f6bfe438da537ac030a8a0d" exitCode=0
Aug 04 15:24:07.223976 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.223892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f67848fb-5fhtq" event={"ID":"d99bca71-914e-4480-84be-de2e808ae294","Type":"ContainerDied","Data":"732beee85065aef5295199cd5a962bd39d6902013f6bfe438da537ac030a8a0d"}
Aug 04 15:24:07.358543 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.358479    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:24:07.481033 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.480993    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g94sm\" (UniqueName: \"kubernetes.io/projected/d99bca71-914e-4480-84be-de2e808ae294-kube-api-access-g94sm\") pod \"d99bca71-914e-4480-84be-de2e808ae294\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") "
Aug 04 15:24:07.481208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.481123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d99bca71-914e-4480-84be-de2e808ae294-tls-cert\") pod \"d99bca71-914e-4480-84be-de2e808ae294\" (UID: \"d99bca71-914e-4480-84be-de2e808ae294\") "
Aug 04 15:24:07.483298 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.483262    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99bca71-914e-4480-84be-de2e808ae294-kube-api-access-g94sm" (OuterVolumeSpecName: "kube-api-access-g94sm") pod "d99bca71-914e-4480-84be-de2e808ae294" (UID: "d99bca71-914e-4480-84be-de2e808ae294"). InnerVolumeSpecName "kube-api-access-g94sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:24:07.492466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.492435    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99bca71-914e-4480-84be-de2e808ae294-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d99bca71-914e-4480-84be-de2e808ae294" (UID: "d99bca71-914e-4480-84be-de2e808ae294"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:24:07.582149 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.582110    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d99bca71-914e-4480-84be-de2e808ae294-tls-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:24:07.582333 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:07.582162    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g94sm\" (UniqueName: \"kubernetes.io/projected/d99bca71-914e-4480-84be-de2e808ae294-kube-api-access-g94sm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:24:08.229886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:08.229790    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f67848fb-5fhtq" event={"ID":"d99bca71-914e-4480-84be-de2e808ae294","Type":"ContainerDied","Data":"2a9df5153b03bd62bb5b7053fba822c39742c8278589ea705977a69bfbbdd3e8"}
Aug 04 15:24:08.229886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:08.229814    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f67848fb-5fhtq"
Aug 04 15:24:08.229886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:08.229838    2569 scope.go:117] "RemoveContainer" containerID="732beee85065aef5295199cd5a962bd39d6902013f6bfe438da537ac030a8a0d"
Aug 04 15:24:08.247073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:08.247041    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f67848fb-5fhtq"]
Aug 04 15:24:08.252801 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:08.252775    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f67848fb-5fhtq"]
Aug 04 15:24:09.965216 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:09.965182    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d99bca71-914e-4480-84be-de2e808ae294" path="/var/lib/kubelet/pods/d99bca71-914e-4480-84be-de2e808ae294/volumes"
Aug 04 15:24:36.220678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:36.220568    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:24:45.814823 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:45.814785    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:24:57.623377 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:24:57.623332    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:26:37.548527 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:26:37.548488    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:26:46.121055 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:26:46.121022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:27:43.919913 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:27:43.919874    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:27:50.644010 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:27:50.643976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:04.799221 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.799182    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74cc45dc7-qbb88"]
Aug 04 15:28:04.799760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.799695    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d99bca71-914e-4480-84be-de2e808ae294" containerName="authorino"
Aug 04 15:28:04.799760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.799714    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d99bca71-914e-4480-84be-de2e808ae294" containerName="authorino"
Aug 04 15:28:04.799889 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.799819    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d99bca71-914e-4480-84be-de2e808ae294" containerName="authorino"
Aug 04 15:28:04.801563 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.801544    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.804596 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.804575    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Aug 04 15:28:04.804833 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.804820    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Aug 04 15:28:04.805664 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.805626    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Aug 04 15:28:04.805783 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.805651    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Aug 04 15:28:04.805855 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.805700    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Aug 04 15:28:04.805855 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.805737    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Aug 04 15:28:04.805958 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.805895    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Aug 04 15:28:04.806148 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.806132    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lzz5b\""
Aug 04 15:28:04.812951 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.812932    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Aug 04 15:28:04.831110 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.831064    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cc45dc7-qbb88"]
Aug 04 15:28:04.888189 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-service-ca\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.888189 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-oauth-config\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.888446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-config\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.888446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89f6b\" (UniqueName: \"kubernetes.io/projected/516efb51-fbb4-4d41-b9f5-5cbbc756749c-kube-api-access-89f6b\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.888446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-oauth-serving-cert\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.888446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-serving-cert\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.888598 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.888457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-trusted-ca-bundle\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.988829 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.988786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-service-ca\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.988829 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.988832    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-oauth-config\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.988854    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-config\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.988877    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89f6b\" (UniqueName: \"kubernetes.io/projected/516efb51-fbb4-4d41-b9f5-5cbbc756749c-kube-api-access-89f6b\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.988918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-oauth-serving-cert\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.988976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-serving-cert\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.989109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-trusted-ca-bundle\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989663 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.989623    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-config\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989761 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.989694    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-oauth-serving-cert\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989761 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.989708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-service-ca\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.989960 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.989943    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/516efb51-fbb4-4d41-b9f5-5cbbc756749c-trusted-ca-bundle\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.992000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.991969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-serving-cert\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.992087 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.991969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/516efb51-fbb4-4d41-b9f5-5cbbc756749c-console-oauth-config\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:04.997188 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:04.997170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89f6b\" (UniqueName: \"kubernetes.io/projected/516efb51-fbb4-4d41-b9f5-5cbbc756749c-kube-api-access-89f6b\") pod \"console-74cc45dc7-qbb88\" (UID: \"516efb51-fbb4-4d41-b9f5-5cbbc756749c\") " pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:05.111237 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:05.111149    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:05.249779 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:05.249752    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cc45dc7-qbb88"]
Aug 04 15:28:05.252308 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:28:05.252267    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516efb51_fbb4_4d41_b9f5_5cbbc756749c.slice/crio-55961fa1169111e2a7aff16cec14abb3fbc3853530b397941f5b8a4506ae3058 WatchSource:0}: Error finding container 55961fa1169111e2a7aff16cec14abb3fbc3853530b397941f5b8a4506ae3058: Status 404 returned error can't find the container with id 55961fa1169111e2a7aff16cec14abb3fbc3853530b397941f5b8a4506ae3058
Aug 04 15:28:06.122267 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:06.122234    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:06.249848 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:06.249810    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cc45dc7-qbb88" event={"ID":"516efb51-fbb4-4d41-b9f5-5cbbc756749c","Type":"ContainerStarted","Data":"709b8e704837a2d5f850de8e8a785749e26abfa7f596b69fd9ef6b8a2f012fb4"}
Aug 04 15:28:06.250031 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:06.249864    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cc45dc7-qbb88" event={"ID":"516efb51-fbb4-4d41-b9f5-5cbbc756749c","Type":"ContainerStarted","Data":"55961fa1169111e2a7aff16cec14abb3fbc3853530b397941f5b8a4506ae3058"}
Aug 04 15:28:06.270365 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:06.270305    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74cc45dc7-qbb88" podStartSLOduration=2.2702841129999998 podStartE2EDuration="2.270284113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:28:06.268666861 +0000 UTC m=+1028.774964911" watchObservedRunningTime="2026-08-04 15:28:06.270284113 +0000 UTC m=+1028.776582266"
Aug 04 15:28:12.917989 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:12.917951    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:15.111429 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:15.111388    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:15.112012 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:15.111453    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:15.116412 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:15.116388    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:15.293084 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:15.293055    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74cc45dc7-qbb88"
Aug 04 15:28:21.836829 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:21.836793    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:28.218757 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:28.218716    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:44.414248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:44.414208    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:49.621537 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:49.621451    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:28:56.523292 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:28:56.523254    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:29:16.130197 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:29:16.130163    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:29:27.054274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:29:27.054237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:29:50.226082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:29:50.226044    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:00.713695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:00.713651    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:05.722486 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:05.722446    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:13.616323 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:13.616282    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:21.625805 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:21.625720    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:26.907454 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:26.907416    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:31.819465 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:31.819425    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:39.221347 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:39.221305    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:53.315552 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:53.315515    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:30:58.509907 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:30:58.509866    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:31:09.118326 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:31:09.118282    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:31:14.424475 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:31:14.424436    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:31:21.022777 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:31:21.022741    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:31:29.416413 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:31:29.416373    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:31:34.621848 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:31:34.621808    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:31:45.420863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:31:45.420821    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:34:00.739999 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:34:00.739962    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:34:05.820770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:34:05.820727    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:34:31.644990 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:34:31.644956    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:35:22.133610 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:35:22.133551    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:35:28.429455 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:35:28.429419    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:35:37.218524 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:35:37.218486    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:35:42.339783 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:35:42.339744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:35:52.921074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:35:52.921027    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:35:59.219125 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:35:59.219089    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:36:08.008794 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:36:08.008754    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:36:13.120749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:36:13.120708    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:36:23.124768 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:36:23.124666    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:36:29.622424 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:36:29.622391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:36:38.514008 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:36:38.513969    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:37:02.123015 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:02.122976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:37:09.016506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:09.016461    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:37:17.414133 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.414095    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"]
Aug 04 15:37:17.417874 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.417848    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.420442 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.420415    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-547eb87b-kserve-self-signed-certs\""
Aug 04 15:37:17.434567 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.434535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e457039-0f76-4155-a193-06bf5622616f-tls-certs\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.434790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.434582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-tmp-dir\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.434790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.434614    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-home\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.434790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.434725    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-model-cache\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.434790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.434761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgvz\" (UniqueName: \"kubernetes.io/projected/8e457039-0f76-4155-a193-06bf5622616f-kube-api-access-mcgvz\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.434790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.434793    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-dshm\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.486216 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.486182    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"]
Aug 04 15:37:17.535607 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.535570    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-tmp-dir\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.535812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.535630    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-home\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.535812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.535697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-model-cache\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.535812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.535717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgvz\" (UniqueName: \"kubernetes.io/projected/8e457039-0f76-4155-a193-06bf5622616f-kube-api-access-mcgvz\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.535812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.535755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-dshm\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.535812 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.535808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e457039-0f76-4155-a193-06bf5622616f-tls-certs\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.536106 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.536079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-tmp-dir\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.536192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.536090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-home\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.536192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.536169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-model-cache\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.538219 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.538191    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-dshm\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.538489 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.538469    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e457039-0f76-4155-a193-06bf5622616f-tls-certs\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.545604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.545576    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgvz\" (UniqueName: \"kubernetes.io/projected/8e457039-0f76-4155-a193-06bf5622616f-kube-api-access-mcgvz\") pod \"e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") " pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.728341 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.728250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:17.870214 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:17.870144    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"]
Aug 04 15:37:17.872772 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:37:17.872743    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e457039_0f76_4155_a193_06bf5622616f.slice/crio-227a385edfe92203812faee74d7a722aec2f09a42739e79b27f63f40ef4ba383 WatchSource:0}: Error finding container 227a385edfe92203812faee74d7a722aec2f09a42739e79b27f63f40ef4ba383: Status 404 returned error can't find the container with id 227a385edfe92203812faee74d7a722aec2f09a42739e79b27f63f40ef4ba383
Aug 04 15:37:18.673757 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:18.673723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n" event={"ID":"8e457039-0f76-4155-a193-06bf5622616f","Type":"ContainerStarted","Data":"19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0"}
Aug 04 15:37:18.673757 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:18.673764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n" event={"ID":"8e457039-0f76-4155-a193-06bf5622616f","Type":"ContainerStarted","Data":"227a385edfe92203812faee74d7a722aec2f09a42739e79b27f63f40ef4ba383"}
Aug 04 15:37:18.693501 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:18.693449    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n" podStartSLOduration=1.693429563 podStartE2EDuration="1.693429563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:37:18.692682084 +0000 UTC m=+1581.198980142" watchObservedRunningTime="2026-08-04 15:37:18.693429563 +0000 UTC m=+1581.199727621"
Aug 04 15:37:20.325569 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:20.325535    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:37:26.873571 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.873530    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"]
Aug 04 15:37:26.877533 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.877508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:26.881016 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.880990    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-547eb87b-kserve-self-signed-certs\""
Aug 04 15:37:26.897876 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.897850    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"]
Aug 04 15:37:26.923738 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.923692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmtwf\" (UniqueName: \"kubernetes.io/projected/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-kube-api-access-nmtwf\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:26.923738 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.923745    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-home\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:26.923954 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.923775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tmp-dir\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:26.923954 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.923819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-dshm\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:26.923954 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.923841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-model-cache\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:26.923954 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:26.923925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tls-certs\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.025336 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.025292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tls-certs\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.025530 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.025403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmtwf\" (UniqueName: \"kubernetes.io/projected/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-kube-api-access-nmtwf\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.025530 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.025438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-home\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.025530 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.025470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tmp-dir\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.025790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.025530    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-dshm\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.025790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.025577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-model-cache\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.026867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.026296    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-home\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.026867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.026464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tmp-dir\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.026867 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.026777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-model-cache\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.035141 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.035108    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tls-certs\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.036584 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.036561    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-dshm\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.037107 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.037089    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmtwf\" (UniqueName: \"kubernetes.io/projected/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-kube-api-access-nmtwf\") pod \"e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") " pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.187538 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.187431    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:27.330754 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.330720    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"]
Aug 04 15:37:27.333672 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:37:27.333614    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb32d3ed3_d174_4cc2_ad53_fe78e0c94c38.slice/crio-d6f97cbe9477f86024873ad7b910b12e596efb6a43b552f93b81614a8cff132a WatchSource:0}: Error finding container d6f97cbe9477f86024873ad7b910b12e596efb6a43b552f93b81614a8cff132a: Status 404 returned error can't find the container with id d6f97cbe9477f86024873ad7b910b12e596efb6a43b552f93b81614a8cff132a
Aug 04 15:37:27.716326 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.716283    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm" event={"ID":"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38","Type":"ContainerStarted","Data":"ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e"}
Aug 04 15:37:27.716511 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.716335    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm" event={"ID":"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38","Type":"ContainerStarted","Data":"d6f97cbe9477f86024873ad7b910b12e596efb6a43b552f93b81614a8cff132a"}
Aug 04 15:37:27.729408 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.729366    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:27.729569 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.729439    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:27.736091 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.736029    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm" podStartSLOduration=1.7360055239999999 podStartE2EDuration="1.736005524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:37:27.734924937 +0000 UTC m=+1590.241222995" watchObservedRunningTime="2026-08-04 15:37:27.736005524 +0000 UTC m=+1590.242303582"
Aug 04 15:37:27.743457 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:27.743428    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:28.731624 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:28.731595    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:37:30.429693 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:30.429652    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:37:37.187793 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:37.187756    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:37.188366 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:37.187835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:37.200892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:37.200861    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:37:37.770606 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:37:37.770571    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:38:03.129622 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:03.129524    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"]
Aug 04 15:38:05.015120 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:05.015087    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:38:05.092441 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:05.092403    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"]
Aug 04 15:38:14.730906 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:14.730866    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:38:18.148932 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.148884    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n" podUID="8e457039-0f76-4155-a193-06bf5622616f" containerName="main" containerID="cri-o://19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0" gracePeriod=45
Aug 04 15:38:18.403835 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.403760    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:38:18.505383 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505351    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-home\") pod \"8e457039-0f76-4155-a193-06bf5622616f\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") "
Aug 04 15:38:18.505383 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505388    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-model-cache\") pod \"8e457039-0f76-4155-a193-06bf5622616f\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") "
Aug 04 15:38:18.505695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505444    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-tmp-dir\") pod \"8e457039-0f76-4155-a193-06bf5622616f\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") "
Aug 04 15:38:18.505695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505478    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-dshm\") pod \"8e457039-0f76-4155-a193-06bf5622616f\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") "
Aug 04 15:38:18.505695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505556    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e457039-0f76-4155-a193-06bf5622616f-tls-certs\") pod \"8e457039-0f76-4155-a193-06bf5622616f\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") "
Aug 04 15:38:18.505695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505585    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcgvz\" (UniqueName: \"kubernetes.io/projected/8e457039-0f76-4155-a193-06bf5622616f-kube-api-access-mcgvz\") pod \"8e457039-0f76-4155-a193-06bf5622616f\" (UID: \"8e457039-0f76-4155-a193-06bf5622616f\") "
Aug 04 15:38:18.505930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505712    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e457039-0f76-4155-a193-06bf5622616f" (UID: "8e457039-0f76-4155-a193-06bf5622616f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:18.505930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505748    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-home" (OuterVolumeSpecName: "home") pod "8e457039-0f76-4155-a193-06bf5622616f" (UID: "8e457039-0f76-4155-a193-06bf5622616f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:18.505930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505794    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e457039-0f76-4155-a193-06bf5622616f" (UID: "8e457039-0f76-4155-a193-06bf5622616f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:18.506082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505943    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:18.506082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505964    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:18.506082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.505977    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:18.507949 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.507916    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-dshm" (OuterVolumeSpecName: "dshm") pod "8e457039-0f76-4155-a193-06bf5622616f" (UID: "8e457039-0f76-4155-a193-06bf5622616f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:18.508301 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.508278    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e457039-0f76-4155-a193-06bf5622616f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e457039-0f76-4155-a193-06bf5622616f" (UID: "8e457039-0f76-4155-a193-06bf5622616f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:38:18.508301 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.508290    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e457039-0f76-4155-a193-06bf5622616f-kube-api-access-mcgvz" (OuterVolumeSpecName: "kube-api-access-mcgvz") pod "8e457039-0f76-4155-a193-06bf5622616f" (UID: "8e457039-0f76-4155-a193-06bf5622616f"). InnerVolumeSpecName "kube-api-access-mcgvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:38:18.606666 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.606595    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e457039-0f76-4155-a193-06bf5622616f-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:18.606666 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.606626    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcgvz\" (UniqueName: \"kubernetes.io/projected/8e457039-0f76-4155-a193-06bf5622616f-kube-api-access-mcgvz\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:18.606666 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.606656    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e457039-0f76-4155-a193-06bf5622616f-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:18.944038 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.944006    2569 generic.go:356] "Generic (PLEG): container finished" podID="8e457039-0f76-4155-a193-06bf5622616f" containerID="19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0" exitCode=0
Aug 04 15:38:18.944223 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.944056    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"
Aug 04 15:38:18.944223 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.944099    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n" event={"ID":"8e457039-0f76-4155-a193-06bf5622616f","Type":"ContainerDied","Data":"19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0"}
Aug 04 15:38:18.944223 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.944147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n" event={"ID":"8e457039-0f76-4155-a193-06bf5622616f","Type":"ContainerDied","Data":"227a385edfe92203812faee74d7a722aec2f09a42739e79b27f63f40ef4ba383"}
Aug 04 15:38:18.944223 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.944165    2569 scope.go:117] "RemoveContainer" containerID="19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0"
Aug 04 15:38:18.953047 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.953029    2569 scope.go:117] "RemoveContainer" containerID="19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0"
Aug 04 15:38:18.953362 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:38:18.953343    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0\": container with ID starting with 19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0 not found: ID does not exist" containerID="19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0"
Aug 04 15:38:18.953410 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.953372    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0"} err="failed to get container status \"19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0\": rpc error: code = NotFound desc = could not find container \"19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0\": container with ID starting with 19a4862ad832bb2f96607a0c8c70aeb1b9dd3ca88956ee504bfc726b010ff7e0 not found: ID does not exist"
Aug 04 15:38:18.966700 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.966666    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"]
Aug 04 15:38:18.971493 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:18.971463    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-547eb87b-kserve-d8dbb8f59-nd76n"]
Aug 04 15:38:19.964418 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:19.964379    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e457039-0f76-4155-a193-06bf5622616f" path="/var/lib/kubelet/pods/8e457039-0f76-4155-a193-06bf5622616f/volumes"
Aug 04 15:38:20.112283 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.112237    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm" podUID="b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" containerName="main" containerID="cri-o://ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e" gracePeriod=45
Aug 04 15:38:20.377954 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.377928    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:38:20.421497 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421466    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-model-cache\") pod \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") "
Aug 04 15:38:20.421705 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421560    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-home\") pod \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") "
Aug 04 15:38:20.421705 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tls-certs\") pod \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") "
Aug 04 15:38:20.421705 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421603    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tmp-dir\") pod \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") "
Aug 04 15:38:20.421705 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421654    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmtwf\" (UniqueName: \"kubernetes.io/projected/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-kube-api-access-nmtwf\") pod \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") "
Aug 04 15:38:20.421930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421711    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-dshm\") pod \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\" (UID: \"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38\") "
Aug 04 15:38:20.421930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-model-cache" (OuterVolumeSpecName: "model-cache") pod "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" (UID: "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:20.421930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421858    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-home" (OuterVolumeSpecName: "home") pod "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" (UID: "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:20.421930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.421890    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" (UID: "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:20.422111 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.422007    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:20.422111 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.422020    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:20.422111 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.422046    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:20.423982 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.423958    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-dshm" (OuterVolumeSpecName: "dshm") pod "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" (UID: "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:38:20.423982 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.423967    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" (UID: "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:38:20.424095 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.423980    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-kube-api-access-nmtwf" (OuterVolumeSpecName: "kube-api-access-nmtwf") pod "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" (UID: "b32d3ed3-d174-4cc2-ad53-fe78e0c94c38"). InnerVolumeSpecName "kube-api-access-nmtwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:38:20.523075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.522986    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:20.523075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.523020    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmtwf\" (UniqueName: \"kubernetes.io/projected/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-kube-api-access-nmtwf\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:20.523075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.523031    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:38:20.953809 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.953768    2569 generic.go:356] "Generic (PLEG): container finished" podID="b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" containerID="ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e" exitCode=0
Aug 04 15:38:20.953991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.953817    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"
Aug 04 15:38:20.953991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.953838    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm" event={"ID":"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38","Type":"ContainerDied","Data":"ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e"}
Aug 04 15:38:20.953991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.953877    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm" event={"ID":"b32d3ed3-d174-4cc2-ad53-fe78e0c94c38","Type":"ContainerDied","Data":"d6f97cbe9477f86024873ad7b910b12e596efb6a43b552f93b81614a8cff132a"}
Aug 04 15:38:20.953991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.953899    2569 scope.go:117] "RemoveContainer" containerID="ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e"
Aug 04 15:38:20.962980 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.962963    2569 scope.go:117] "RemoveContainer" containerID="ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e"
Aug 04 15:38:20.963248 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:38:20.963233    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e\": container with ID starting with ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e not found: ID does not exist" containerID="ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e"
Aug 04 15:38:20.963292 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.963260    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e"} err="failed to get container status \"ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e\": rpc error: code = NotFound desc = could not find container \"ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e\": container with ID starting with ceb54f610d217dfa38b6eb73cdbd21851a333fe2ff898ab6776d8daaf83f7a0e not found: ID does not exist"
Aug 04 15:38:20.980986 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.980955    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"]
Aug 04 15:38:20.985383 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:20.985355    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-547eb87b-kserve-77cd64d59-rc2hm"]
Aug 04 15:38:21.963817 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:21.963784    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" path="/var/lib/kubelet/pods/b32d3ed3-d174-4cc2-ad53-fe78e0c94c38/volumes"
Aug 04 15:38:22.527488 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:22.527453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:38:29.035868 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:29.035824    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:38:39.619302 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:39.619257    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:38:47.024017 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:47.023976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:38:57.217169 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:38:57.217131    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:39:04.008243 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:39:04.008212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:39:14.114073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:39:14.114035    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:39:21.929172 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:39:21.929084    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:39:36.825564 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:39:36.825524    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:39:43.716762 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:39:43.716724    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:39:55.636600 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:39:55.636560    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:03.011251 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:03.011216    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:09.836628 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:09.836588    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:14.631891 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:14.631852    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:21.334012 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:21.333973    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:30.835105 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:30.835063    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:38.355579 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:38.355549    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:48.224680 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:48.224615    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:40:54.632131 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:40:54.632041    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:05.556995 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:05.556956    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:20.730310 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:20.730280    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:27.873403 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:27.873364    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:38.419942 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:38.419906    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:45.131263 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:45.131222    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:53.322290 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:53.322256    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:41:59.120739 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:41:59.120705    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:42:14.334361 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:14.334321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:42:35.175899 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.175816    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"]
Aug 04 15:42:35.176373 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.176147    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e457039-0f76-4155-a193-06bf5622616f" containerName="main"
Aug 04 15:42:35.176373 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.176158    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e457039-0f76-4155-a193-06bf5622616f" containerName="main"
Aug 04 15:42:35.176373 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.176174    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" containerName="main"
Aug 04 15:42:35.176373 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.176179    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" containerName="main"
Aug 04 15:42:35.176373 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.176230    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32d3ed3-d174-4cc2-ad53-fe78e0c94c38" containerName="main"
Aug 04 15:42:35.176373 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.176240    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e457039-0f76-4155-a193-06bf5622616f" containerName="main"
Aug 04 15:42:35.178873 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.178854    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:35.181496 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.181472    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:42:35.181632 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.181573    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:42:35.189739 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.189710    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"]
Aug 04 15:42:35.242083 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.242039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/399cdf74-4732-4627-aa76-f3de01aca51f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-197040d2-899xm\" (UID: \"399cdf74-4732-4627-aa76-f3de01aca51f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:35.343430 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.343392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/399cdf74-4732-4627-aa76-f3de01aca51f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-197040d2-899xm\" (UID: \"399cdf74-4732-4627-aa76-f3de01aca51f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:35.344083 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.344061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/399cdf74-4732-4627-aa76-f3de01aca51f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-197040d2-899xm\" (UID: \"399cdf74-4732-4627-aa76-f3de01aca51f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:35.490709 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.490592    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:35.628171 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.628137    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"]
Aug 04 15:42:35.630908 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:42:35.630876    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399cdf74_4732_4627_aa76_f3de01aca51f.slice/crio-24e0685fab468b8e107fc9c537d02598ecb56e0d36d9c03311547a9341d34ff8 WatchSource:0}: Error finding container 24e0685fab468b8e107fc9c537d02598ecb56e0d36d9c03311547a9341d34ff8: Status 404 returned error can't find the container with id 24e0685fab468b8e107fc9c537d02598ecb56e0d36d9c03311547a9341d34ff8
Aug 04 15:42:35.632926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:35.632910    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 15:42:36.055186 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:36.055145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm" event={"ID":"399cdf74-4732-4627-aa76-f3de01aca51f","Type":"ContainerStarted","Data":"24e0685fab468b8e107fc9c537d02598ecb56e0d36d9c03311547a9341d34ff8"}
Aug 04 15:42:38.065956 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:38.065917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm" event={"ID":"399cdf74-4732-4627-aa76-f3de01aca51f","Type":"ContainerStarted","Data":"4fb9b69862a6379d81b86419f5d80e22bf6e80c144b15d1492c1bdd39b2ccdb1"}
Aug 04 15:42:38.084052 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:38.083916    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm" podStartSLOduration=1.134329525 podStartE2EDuration="3.083895911s" totalImagesPullingTime="1.949566386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:42:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:42:38.082731311 +0000 UTC m=+1900.589029369" watchObservedRunningTime="2026-08-04 15:42:38.083895911 +0000 UTC m=+1900.590193967"
Aug 04 15:42:39.072565 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:39.072531    2569 generic.go:356] "Generic (PLEG): container finished" podID="399cdf74-4732-4627-aa76-f3de01aca51f" containerID="4fb9b69862a6379d81b86419f5d80e22bf6e80c144b15d1492c1bdd39b2ccdb1" exitCode=0
Aug 04 15:42:39.072995 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:39.072586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm" event={"ID":"399cdf74-4732-4627-aa76-f3de01aca51f","Type":"ContainerDied","Data":"4fb9b69862a6379d81b86419f5d80e22bf6e80c144b15d1492c1bdd39b2ccdb1"}
Aug 04 15:42:40.210790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:40.210758    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:40.288854 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:40.288794    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/399cdf74-4732-4627-aa76-f3de01aca51f-maas-api-service-ca\") pod \"399cdf74-4732-4627-aa76-f3de01aca51f\" (UID: \"399cdf74-4732-4627-aa76-f3de01aca51f\") "
Aug 04 15:42:40.289177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:40.289150    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/399cdf74-4732-4627-aa76-f3de01aca51f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "399cdf74-4732-4627-aa76-f3de01aca51f" (UID: "399cdf74-4732-4627-aa76-f3de01aca51f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:42:40.390571 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:40.390532    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/399cdf74-4732-4627-aa76-f3de01aca51f-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:42:41.083405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:41.083366    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm" event={"ID":"399cdf74-4732-4627-aa76-f3de01aca51f","Type":"ContainerDied","Data":"24e0685fab468b8e107fc9c537d02598ecb56e0d36d9c03311547a9341d34ff8"}
Aug 04 15:42:41.083405 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:41.083393    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"
Aug 04 15:42:41.083623 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:41.083418    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24e0685fab468b8e107fc9c537d02598ecb56e0d36d9c03311547a9341d34ff8"
Aug 04 15:42:51.534379 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:51.534335    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"]
Aug 04 15:42:51.536278 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:51.536253    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-197040d2-899xm"]
Aug 04 15:42:51.964113 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:51.964078    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="399cdf74-4732-4627-aa76-f3de01aca51f" path="/var/lib/kubelet/pods/399cdf74-4732-4627-aa76-f3de01aca51f/volumes"
Aug 04 15:42:54.316795 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:42:54.316758    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:43:01.720122 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:01.720087    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:43:19.964839 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.964806    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"]
Aug 04 15:43:19.965276 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.965115    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="399cdf74-4732-4627-aa76-f3de01aca51f" containerName="revoke-keys"
Aug 04 15:43:19.965276 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.965127    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="399cdf74-4732-4627-aa76-f3de01aca51f" containerName="revoke-keys"
Aug 04 15:43:19.965276 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.965181    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="399cdf74-4732-4627-aa76-f3de01aca51f" containerName="revoke-keys"
Aug 04 15:43:19.973086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.973049    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"]
Aug 04 15:43:19.973265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.973157    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:19.975418 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.975388    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:43:19.975575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:19.975526    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:43:20.069095 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:20.069062    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c787fba-0542-4a21-9a8e-64d7e6870435-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4\" (UID: \"5c787fba-0542-4a21-9a8e-64d7e6870435\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:20.170136 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:20.170091    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c787fba-0542-4a21-9a8e-64d7e6870435-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4\" (UID: \"5c787fba-0542-4a21-9a8e-64d7e6870435\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:20.170786 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:20.170764    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c787fba-0542-4a21-9a8e-64d7e6870435-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4\" (UID: \"5c787fba-0542-4a21-9a8e-64d7e6870435\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:20.284799 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:20.284705    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:20.423538 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:20.423510    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"]
Aug 04 15:43:20.426387 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:43:20.426360    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c787fba_0542_4a21_9a8e_64d7e6870435.slice/crio-acc913807aaf735d4dd8b6edca13ea2a3ce424c0f2f955c498d46b22ded06dd7 WatchSource:0}: Error finding container acc913807aaf735d4dd8b6edca13ea2a3ce424c0f2f955c498d46b22ded06dd7: Status 404 returned error can't find the container with id acc913807aaf735d4dd8b6edca13ea2a3ce424c0f2f955c498d46b22ded06dd7
Aug 04 15:43:21.261747 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:21.261712    2569 generic.go:356] "Generic (PLEG): container finished" podID="5c787fba-0542-4a21-9a8e-64d7e6870435" containerID="0583a93f0306f735ce700c550c8b970334d30ffa7cbf49f33414458678754731" exitCode=0
Aug 04 15:43:21.262172 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:21.261804    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4" event={"ID":"5c787fba-0542-4a21-9a8e-64d7e6870435","Type":"ContainerDied","Data":"0583a93f0306f735ce700c550c8b970334d30ffa7cbf49f33414458678754731"}
Aug 04 15:43:21.262172 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:21.261852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4" event={"ID":"5c787fba-0542-4a21-9a8e-64d7e6870435","Type":"ContainerStarted","Data":"acc913807aaf735d4dd8b6edca13ea2a3ce424c0f2f955c498d46b22ded06dd7"}
Aug 04 15:43:22.399466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:22.399437    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:22.489261 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:22.489230    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c787fba-0542-4a21-9a8e-64d7e6870435-maas-api-service-ca\") pod \"5c787fba-0542-4a21-9a8e-64d7e6870435\" (UID: \"5c787fba-0542-4a21-9a8e-64d7e6870435\") "
Aug 04 15:43:22.489667 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:22.489624    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c787fba-0542-4a21-9a8e-64d7e6870435-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5c787fba-0542-4a21-9a8e-64d7e6870435" (UID: "5c787fba-0542-4a21-9a8e-64d7e6870435"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:43:22.590214 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:22.590121    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c787fba-0542-4a21-9a8e-64d7e6870435-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:43:23.272886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:23.272859    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"
Aug 04 15:43:23.273075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:23.272889    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4" event={"ID":"5c787fba-0542-4a21-9a8e-64d7e6870435","Type":"ContainerDied","Data":"acc913807aaf735d4dd8b6edca13ea2a3ce424c0f2f955c498d46b22ded06dd7"}
Aug 04 15:43:23.273075 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:23.272931    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acc913807aaf735d4dd8b6edca13ea2a3ce424c0f2f955c498d46b22ded06dd7"
Aug 04 15:43:26.738862 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:26.738827    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"]
Aug 04 15:43:26.742365 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:26.742338    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-25b12afe-s5ww4"]
Aug 04 15:43:27.965122 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:27.965087    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c787fba-0542-4a21-9a8e-64d7e6870435" path="/var/lib/kubelet/pods/5c787fba-0542-4a21-9a8e-64d7e6870435/volumes"
Aug 04 15:43:29.323961 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:29.323923    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:43:31.896177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.896113    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"]
Aug 04 15:43:31.896560 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.896499    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c787fba-0542-4a21-9a8e-64d7e6870435" containerName="revoke-keys"
Aug 04 15:43:31.896560 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.896511    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c787fba-0542-4a21-9a8e-64d7e6870435" containerName="revoke-keys"
Aug 04 15:43:31.896651 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.896575    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c787fba-0542-4a21-9a8e-64d7e6870435" containerName="revoke-keys"
Aug 04 15:43:31.899564 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.899538    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:31.901943 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.901920    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-018493d3-dockercfg-lbwzb\""
Aug 04 15:43:31.902398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.902377    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-018493d3\""
Aug 04 15:43:31.911108 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:31.911077    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"]
Aug 04 15:43:32.074924 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.074856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.075122 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.075059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d8fr\" (UniqueName: \"kubernetes.io/projected/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-kube-api-access-8d8fr\") pod \"payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.175902 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.175801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d8fr\" (UniqueName: \"kubernetes.io/projected/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-kube-api-access-8d8fr\") pod \"payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.175902 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.175871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.176607 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.176587    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.184524 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.184488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d8fr\" (UniqueName: \"kubernetes.io/projected/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-kube-api-access-8d8fr\") pod \"payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.210149 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.210112    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:32.347112 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:32.346990    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"]
Aug 04 15:43:32.349948 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:43:32.349916    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfcfdcd_8428_432f_8110_8c8ac4a59b61.slice/crio-e2e30900ea1e9e03e669796a75c63bfcf10a9ad782efb73b36b1b39185a134ae WatchSource:0}: Error finding container e2e30900ea1e9e03e669796a75c63bfcf10a9ad782efb73b36b1b39185a134ae: Status 404 returned error can't find the container with id e2e30900ea1e9e03e669796a75c63bfcf10a9ad782efb73b36b1b39185a134ae
Aug 04 15:43:33.320652 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:33.320611    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw" event={"ID":"3cfcfdcd-8428-432f-8110-8c8ac4a59b61","Type":"ContainerStarted","Data":"e2e30900ea1e9e03e669796a75c63bfcf10a9ad782efb73b36b1b39185a134ae"}
Aug 04 15:43:34.327149 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:34.327116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw" event={"ID":"3cfcfdcd-8428-432f-8110-8c8ac4a59b61","Type":"ContainerStarted","Data":"1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449"}
Aug 04 15:43:34.327625 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:34.327327    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:34.345957 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:34.345897    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw" podStartSLOduration=1.552837023 podStartE2EDuration="3.345876152s" totalImagesPullingTime="1.793039129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:43:34.343865066 +0000 UTC m=+1956.850163122" watchObservedRunningTime="2026-08-04 15:43:34.345876152 +0000 UTC m=+1956.852174247"
Aug 04 15:43:37.213651 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:37.213601    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:43:45.333323 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:45.333292    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:43:55.874612 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:55.874525    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"]
Aug 04 15:43:55.879151 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:55.879127    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:55.881807 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:55.881780    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:43:55.882138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:55.882118    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:43:55.887802 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:55.887771    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"]
Aug 04 15:43:55.974448 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:55.974408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00214e2e-9583-43ec-8ade-65bf35de66fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-018493d3-n474v\" (UID: \"00214e2e-9583-43ec-8ade-65bf35de66fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:56.075706 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:56.075634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00214e2e-9583-43ec-8ade-65bf35de66fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-018493d3-n474v\" (UID: \"00214e2e-9583-43ec-8ade-65bf35de66fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:56.076335 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:56.076315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00214e2e-9583-43ec-8ade-65bf35de66fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-018493d3-n474v\" (UID: \"00214e2e-9583-43ec-8ade-65bf35de66fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:56.191877 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:56.191778    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:56.334590 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:56.334564    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"]
Aug 04 15:43:56.337561 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:43:56.337515    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00214e2e_9583_43ec_8ade_65bf35de66fb.slice/crio-44cbddcbcd1cd15fa2e8cbe08a9e41d3844ec8e3b695349d447e7c3fa9a02e73 WatchSource:0}: Error finding container 44cbddcbcd1cd15fa2e8cbe08a9e41d3844ec8e3b695349d447e7c3fa9a02e73: Status 404 returned error can't find the container with id 44cbddcbcd1cd15fa2e8cbe08a9e41d3844ec8e3b695349d447e7c3fa9a02e73
Aug 04 15:43:56.423796 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:56.423764    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v" event={"ID":"00214e2e-9583-43ec-8ade-65bf35de66fb","Type":"ContainerStarted","Data":"44cbddcbcd1cd15fa2e8cbe08a9e41d3844ec8e3b695349d447e7c3fa9a02e73"}
Aug 04 15:43:57.430255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:57.430221    2569 generic.go:356] "Generic (PLEG): container finished" podID="00214e2e-9583-43ec-8ade-65bf35de66fb" containerID="c3d87ffdb5eab8a8e822e996cd6b638c8961c9e4569df367d378859844a93730" exitCode=0
Aug 04 15:43:57.430740 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:57.430289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v" event={"ID":"00214e2e-9583-43ec-8ade-65bf35de66fb","Type":"ContainerDied","Data":"c3d87ffdb5eab8a8e822e996cd6b638c8961c9e4569df367d378859844a93730"}
Aug 04 15:43:58.585236 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:58.585210    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:58.696886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:58.696772    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00214e2e-9583-43ec-8ade-65bf35de66fb-maas-api-service-ca\") pod \"00214e2e-9583-43ec-8ade-65bf35de66fb\" (UID: \"00214e2e-9583-43ec-8ade-65bf35de66fb\") "
Aug 04 15:43:58.697177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:58.697149    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00214e2e-9583-43ec-8ade-65bf35de66fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00214e2e-9583-43ec-8ade-65bf35de66fb" (UID: "00214e2e-9583-43ec-8ade-65bf35de66fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:43:58.798076 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:58.798020    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00214e2e-9583-43ec-8ade-65bf35de66fb-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:43:59.440305 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:59.440276    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"
Aug 04 15:43:59.440490 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:59.440274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v" event={"ID":"00214e2e-9583-43ec-8ade-65bf35de66fb","Type":"ContainerDied","Data":"44cbddcbcd1cd15fa2e8cbe08a9e41d3844ec8e3b695349d447e7c3fa9a02e73"}
Aug 04 15:43:59.440490 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:43:59.440396    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44cbddcbcd1cd15fa2e8cbe08a9e41d3844ec8e3b695349d447e7c3fa9a02e73"
Aug 04 15:44:01.848445 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:01.848400    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"]
Aug 04 15:44:01.848903 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:01.848628    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw" podUID="3cfcfdcd-8428-432f-8110-8c8ac4a59b61" containerName="payload-pre-processing" containerID="cri-o://1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449" gracePeriod=30
Aug 04 15:44:02.029732 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.027796    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"]
Aug 04 15:44:02.035804 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.035768    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-018493d3-n474v"]
Aug 04 15:44:02.103107 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.103034    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:44:02.233173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.233132    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d8fr\" (UniqueName: \"kubernetes.io/projected/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-kube-api-access-8d8fr\") pod \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") "
Aug 04 15:44:02.233358 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.233197    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-plugins-config-volume\") pod \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\" (UID: \"3cfcfdcd-8428-432f-8110-8c8ac4a59b61\") "
Aug 04 15:44:02.233562 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.233532    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cfcfdcd-8428-432f-8110-8c8ac4a59b61" (UID: "3cfcfdcd-8428-432f-8110-8c8ac4a59b61"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:44:02.235416 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.235395    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-kube-api-access-8d8fr" (OuterVolumeSpecName: "kube-api-access-8d8fr") pod "3cfcfdcd-8428-432f-8110-8c8ac4a59b61" (UID: "3cfcfdcd-8428-432f-8110-8c8ac4a59b61"). InnerVolumeSpecName "kube-api-access-8d8fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:44:02.334458 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.334421    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d8fr\" (UniqueName: \"kubernetes.io/projected/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-kube-api-access-8d8fr\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:02.334458 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.334450    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfcfdcd-8428-432f-8110-8c8ac4a59b61-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:02.454485 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.454448    2569 generic.go:356] "Generic (PLEG): container finished" podID="3cfcfdcd-8428-432f-8110-8c8ac4a59b61" containerID="1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449" exitCode=0
Aug 04 15:44:02.454485 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.454493    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"
Aug 04 15:44:02.454791 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.454520    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw" event={"ID":"3cfcfdcd-8428-432f-8110-8c8ac4a59b61","Type":"ContainerDied","Data":"1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449"}
Aug 04 15:44:02.454791 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.454575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw" event={"ID":"3cfcfdcd-8428-432f-8110-8c8ac4a59b61","Type":"ContainerDied","Data":"e2e30900ea1e9e03e669796a75c63bfcf10a9ad782efb73b36b1b39185a134ae"}
Aug 04 15:44:02.454791 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.454605    2569 scope.go:117] "RemoveContainer" containerID="1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449"
Aug 04 15:44:02.464141 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.464116    2569 scope.go:117] "RemoveContainer" containerID="1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449"
Aug 04 15:44:02.464423 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:44:02.464405    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449\": container with ID starting with 1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449 not found: ID does not exist" containerID="1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449"
Aug 04 15:44:02.464481 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.464432    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449"} err="failed to get container status \"1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449\": rpc error: code = NotFound desc = could not find container \"1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449\": container with ID starting with 1c9c3b7b083f599409f522080bfde33000582cf8a79ed60fcfb9c2476a830449 not found: ID does not exist"
Aug 04 15:44:02.476361 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.476329    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"]
Aug 04 15:44:02.484219 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:02.484190    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-018493d3-565ffd75d8-7kjvw"]
Aug 04 15:44:03.965004 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:03.964964    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00214e2e-9583-43ec-8ade-65bf35de66fb" path="/var/lib/kubelet/pods/00214e2e-9583-43ec-8ade-65bf35de66fb/volumes"
Aug 04 15:44:03.965402 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:03.965331    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cfcfdcd-8428-432f-8110-8c8ac4a59b61" path="/var/lib/kubelet/pods/3cfcfdcd-8428-432f-8110-8c8ac4a59b61/volumes"
Aug 04 15:44:04.812736 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:04.812699    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:44:08.053087 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053040    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"]
Aug 04 15:44:08.053618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053526    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cfcfdcd-8428-432f-8110-8c8ac4a59b61" containerName="payload-pre-processing"
Aug 04 15:44:08.053618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053554    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfcfdcd-8428-432f-8110-8c8ac4a59b61" containerName="payload-pre-processing"
Aug 04 15:44:08.053618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053570    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00214e2e-9583-43ec-8ade-65bf35de66fb" containerName="revoke-keys"
Aug 04 15:44:08.053618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053579    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="00214e2e-9583-43ec-8ade-65bf35de66fb" containerName="revoke-keys"
Aug 04 15:44:08.053926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053687    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cfcfdcd-8428-432f-8110-8c8ac4a59b61" containerName="payload-pre-processing"
Aug 04 15:44:08.053926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.053708    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="00214e2e-9583-43ec-8ade-65bf35de66fb" containerName="revoke-keys"
Aug 04 15:44:08.058374 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.058350    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.060907 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.060882    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-cde38272-dockercfg-nzzqj\""
Aug 04 15:44:08.061045 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.060957    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-cde38272\""
Aug 04 15:44:08.066028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.066003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"]
Aug 04 15:44:08.069374 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.069356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.079207 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.079176    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"]
Aug 04 15:44:08.092280 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.092238    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"]
Aug 04 15:44:08.187648 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.187592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbd25\" (UniqueName: \"kubernetes.io/projected/691d794d-cbd2-4751-be5a-c5955f08cae7-kube-api-access-kbd25\") pod \"payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.187838 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.187732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ded334d-2483-4e4d-a168-41a6ac0376e1-plugins-config-volume\") pod \"payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") " pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.187838 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.187811    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dcrv\" (UniqueName: \"kubernetes.io/projected/2ded334d-2483-4e4d-a168-41a6ac0376e1-kube-api-access-8dcrv\") pod \"payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") " pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.187838 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.187831    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/691d794d-cbd2-4751-be5a-c5955f08cae7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.288880 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.288837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbd25\" (UniqueName: \"kubernetes.io/projected/691d794d-cbd2-4751-be5a-c5955f08cae7-kube-api-access-kbd25\") pod \"payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.289094 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.288914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ded334d-2483-4e4d-a168-41a6ac0376e1-plugins-config-volume\") pod \"payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") " pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.289094 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.288969    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dcrv\" (UniqueName: \"kubernetes.io/projected/2ded334d-2483-4e4d-a168-41a6ac0376e1-kube-api-access-8dcrv\") pod \"payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") " pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.289094 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.288985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/691d794d-cbd2-4751-be5a-c5955f08cae7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.289744 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.289721    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ded334d-2483-4e4d-a168-41a6ac0376e1-plugins-config-volume\") pod \"payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") " pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.289845 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.289767    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/691d794d-cbd2-4751-be5a-c5955f08cae7-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.303340 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.303277    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dcrv\" (UniqueName: \"kubernetes.io/projected/2ded334d-2483-4e4d-a168-41a6ac0376e1-kube-api-access-8dcrv\") pod \"payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") " pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.305633 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.305605    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbd25\" (UniqueName: \"kubernetes.io/projected/691d794d-cbd2-4751-be5a-c5955f08cae7-kube-api-access-kbd25\") pod \"payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.368617 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.368575    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:08.380883 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.380860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:08.524468 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.524369    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"]
Aug 04 15:44:08.526844 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:44:08.526809    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691d794d_cbd2_4751_be5a_c5955f08cae7.slice/crio-7a6a0190778c4e6482801647c2a651f19fc950f51bca9b3c925e1131a2e1674f WatchSource:0}: Error finding container 7a6a0190778c4e6482801647c2a651f19fc950f51bca9b3c925e1131a2e1674f: Status 404 returned error can't find the container with id 7a6a0190778c4e6482801647c2a651f19fc950f51bca9b3c925e1131a2e1674f
Aug 04 15:44:08.543672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:08.543584    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"]
Aug 04 15:44:08.546024 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:44:08.545996    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ded334d_2483_4e4d_a168_41a6ac0376e1.slice/crio-e58251c6ce5246e603bab5a6a0e18f18dc78d2593482ffea54461395885e1799 WatchSource:0}: Error finding container e58251c6ce5246e603bab5a6a0e18f18dc78d2593482ffea54461395885e1799: Status 404 returned error can't find the container with id e58251c6ce5246e603bab5a6a0e18f18dc78d2593482ffea54461395885e1799
Aug 04 15:44:09.490929 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.490888    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp" event={"ID":"2ded334d-2483-4e4d-a168-41a6ac0376e1","Type":"ContainerStarted","Data":"28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d"}
Aug 04 15:44:09.491443 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.490948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp" event={"ID":"2ded334d-2483-4e4d-a168-41a6ac0376e1","Type":"ContainerStarted","Data":"e58251c6ce5246e603bab5a6a0e18f18dc78d2593482ffea54461395885e1799"}
Aug 04 15:44:09.491443 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.490994    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:09.492595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.492568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr" event={"ID":"691d794d-cbd2-4751-be5a-c5955f08cae7","Type":"ContainerStarted","Data":"f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154"}
Aug 04 15:44:09.492723 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.492615    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr" event={"ID":"691d794d-cbd2-4751-be5a-c5955f08cae7","Type":"ContainerStarted","Data":"7a6a0190778c4e6482801647c2a651f19fc950f51bca9b3c925e1131a2e1674f"}
Aug 04 15:44:09.492723 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.492665    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:09.512585 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.512534    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp" podStartSLOduration=1.51251646 podStartE2EDuration="1.51251646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:44:09.510725755 +0000 UTC m=+1992.017023834" watchObservedRunningTime="2026-08-04 15:44:09.51251646 +0000 UTC m=+1992.018814515"
Aug 04 15:44:09.534540 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:09.534468    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr" podStartSLOduration=1.534445087 podStartE2EDuration="1.534445087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:44:09.531167826 +0000 UTC m=+1992.037465886" watchObservedRunningTime="2026-08-04 15:44:09.534445087 +0000 UTC m=+1992.040743144"
Aug 04 15:44:12.714795 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:12.714754    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:44:20.499501 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:20.499468    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:20.499889 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:20.499542    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:31.151532 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.151489    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"]
Aug 04 15:44:31.154885 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.154864    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:31.157229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.157196    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:44:31.157362 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.157241    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:44:31.163556 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.163531    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"]
Aug 04 15:44:31.206935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.206889    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b053fda-62c7-43d2-b527-9c6712de54cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cde38272-22jhj\" (UID: \"0b053fda-62c7-43d2-b527-9c6712de54cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:31.307902 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.307861    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b053fda-62c7-43d2-b527-9c6712de54cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cde38272-22jhj\" (UID: \"0b053fda-62c7-43d2-b527-9c6712de54cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:31.308527 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.308506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b053fda-62c7-43d2-b527-9c6712de54cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-cde38272-22jhj\" (UID: \"0b053fda-62c7-43d2-b527-9c6712de54cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:31.466709 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.466573    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:31.608764 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:31.608731    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"]
Aug 04 15:44:31.613539 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:44:31.613497    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b053fda_62c7_43d2_b527_9c6712de54cb.slice/crio-b3a28a01e31512fe2e503af560006e939363d2860ace0aabbb3ec771092bad5d WatchSource:0}: Error finding container b3a28a01e31512fe2e503af560006e939363d2860ace0aabbb3ec771092bad5d: Status 404 returned error can't find the container with id b3a28a01e31512fe2e503af560006e939363d2860ace0aabbb3ec771092bad5d
Aug 04 15:44:32.608416 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:32.608318    2569 generic.go:356] "Generic (PLEG): container finished" podID="0b053fda-62c7-43d2-b527-9c6712de54cb" containerID="5c817bac5bce2c465f7fe6163a2c5b86da538308f53f8540f76280dea57f349c" exitCode=0
Aug 04 15:44:32.608840 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:32.608409    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj" event={"ID":"0b053fda-62c7-43d2-b527-9c6712de54cb","Type":"ContainerDied","Data":"5c817bac5bce2c465f7fe6163a2c5b86da538308f53f8540f76280dea57f349c"}
Aug 04 15:44:32.608840 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:32.608475    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj" event={"ID":"0b053fda-62c7-43d2-b527-9c6712de54cb","Type":"ContainerStarted","Data":"b3a28a01e31512fe2e503af560006e939363d2860ace0aabbb3ec771092bad5d"}
Aug 04 15:44:33.750585 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:33.750555    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:33.829069 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:33.829032    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b053fda-62c7-43d2-b527-9c6712de54cb-maas-api-service-ca\") pod \"0b053fda-62c7-43d2-b527-9c6712de54cb\" (UID: \"0b053fda-62c7-43d2-b527-9c6712de54cb\") "
Aug 04 15:44:33.829402 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:33.829377    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b053fda-62c7-43d2-b527-9c6712de54cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b053fda-62c7-43d2-b527-9c6712de54cb" (UID: "0b053fda-62c7-43d2-b527-9c6712de54cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:44:33.930069 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:33.929979    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b053fda-62c7-43d2-b527-9c6712de54cb-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:34.619980 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:34.619948    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"
Aug 04 15:44:34.620169 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:34.619976    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj" event={"ID":"0b053fda-62c7-43d2-b527-9c6712de54cb","Type":"ContainerDied","Data":"b3a28a01e31512fe2e503af560006e939363d2860ace0aabbb3ec771092bad5d"}
Aug 04 15:44:34.620169 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:34.620025    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3a28a01e31512fe2e503af560006e939363d2860ace0aabbb3ec771092bad5d"
Aug 04 15:44:38.132078 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.132036    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"]
Aug 04 15:44:38.132523 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.132282    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp" podUID="2ded334d-2483-4e4d-a168-41a6ac0376e1" containerName="payload-processing" containerID="cri-o://28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d" gracePeriod=30
Aug 04 15:44:38.137664 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.137155    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"]
Aug 04 15:44:38.137664 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.137387    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr" podUID="691d794d-cbd2-4751-be5a-c5955f08cae7" containerName="payload-pre-processing" containerID="cri-o://f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154" gracePeriod=30
Aug 04 15:44:38.279453 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.279417    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"]
Aug 04 15:44:38.288147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.288113    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-cde38272-22jhj"]
Aug 04 15:44:38.389268 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.389204    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:38.418045 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.417930    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:38.473717 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.473676    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dcrv\" (UniqueName: \"kubernetes.io/projected/2ded334d-2483-4e4d-a168-41a6ac0376e1-kube-api-access-8dcrv\") pod \"2ded334d-2483-4e4d-a168-41a6ac0376e1\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") "
Aug 04 15:44:38.473926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.473731    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/691d794d-cbd2-4751-be5a-c5955f08cae7-plugins-config-volume\") pod \"691d794d-cbd2-4751-be5a-c5955f08cae7\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") "
Aug 04 15:44:38.473926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.473767    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbd25\" (UniqueName: \"kubernetes.io/projected/691d794d-cbd2-4751-be5a-c5955f08cae7-kube-api-access-kbd25\") pod \"691d794d-cbd2-4751-be5a-c5955f08cae7\" (UID: \"691d794d-cbd2-4751-be5a-c5955f08cae7\") "
Aug 04 15:44:38.473926 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.473807    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ded334d-2483-4e4d-a168-41a6ac0376e1-plugins-config-volume\") pod \"2ded334d-2483-4e4d-a168-41a6ac0376e1\" (UID: \"2ded334d-2483-4e4d-a168-41a6ac0376e1\") "
Aug 04 15:44:38.474218 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.474189    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ded334d-2483-4e4d-a168-41a6ac0376e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2ded334d-2483-4e4d-a168-41a6ac0376e1" (UID: "2ded334d-2483-4e4d-a168-41a6ac0376e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:44:38.474218 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.474204    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/691d794d-cbd2-4751-be5a-c5955f08cae7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "691d794d-cbd2-4751-be5a-c5955f08cae7" (UID: "691d794d-cbd2-4751-be5a-c5955f08cae7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:44:38.476074 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.476049    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/691d794d-cbd2-4751-be5a-c5955f08cae7-kube-api-access-kbd25" (OuterVolumeSpecName: "kube-api-access-kbd25") pod "691d794d-cbd2-4751-be5a-c5955f08cae7" (UID: "691d794d-cbd2-4751-be5a-c5955f08cae7"). InnerVolumeSpecName "kube-api-access-kbd25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:44:38.476191 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.476088    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ded334d-2483-4e4d-a168-41a6ac0376e1-kube-api-access-8dcrv" (OuterVolumeSpecName: "kube-api-access-8dcrv") pod "2ded334d-2483-4e4d-a168-41a6ac0376e1" (UID: "2ded334d-2483-4e4d-a168-41a6ac0376e1"). InnerVolumeSpecName "kube-api-access-8dcrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:44:38.574650 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.574604    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbd25\" (UniqueName: \"kubernetes.io/projected/691d794d-cbd2-4751-be5a-c5955f08cae7-kube-api-access-kbd25\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:38.574650 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.574632    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ded334d-2483-4e4d-a168-41a6ac0376e1-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:38.574870 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.574660    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dcrv\" (UniqueName: \"kubernetes.io/projected/2ded334d-2483-4e4d-a168-41a6ac0376e1-kube-api-access-8dcrv\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:38.574870 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.574669    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/691d794d-cbd2-4751-be5a-c5955f08cae7-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:44:38.643822 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.643716    2569 generic.go:356] "Generic (PLEG): container finished" podID="2ded334d-2483-4e4d-a168-41a6ac0376e1" containerID="28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d" exitCode=0
Aug 04 15:44:38.643822 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.643769    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"
Aug 04 15:44:38.644008 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.643815    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp" event={"ID":"2ded334d-2483-4e4d-a168-41a6ac0376e1","Type":"ContainerDied","Data":"28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d"}
Aug 04 15:44:38.644008 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.643875    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp" event={"ID":"2ded334d-2483-4e4d-a168-41a6ac0376e1","Type":"ContainerDied","Data":"e58251c6ce5246e603bab5a6a0e18f18dc78d2593482ffea54461395885e1799"}
Aug 04 15:44:38.644008 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.643902    2569 scope.go:117] "RemoveContainer" containerID="28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d"
Aug 04 15:44:38.645182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.645159    2569 generic.go:356] "Generic (PLEG): container finished" podID="691d794d-cbd2-4751-be5a-c5955f08cae7" containerID="f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154" exitCode=0
Aug 04 15:44:38.645314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.645205    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"
Aug 04 15:44:38.645314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.645214    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr" event={"ID":"691d794d-cbd2-4751-be5a-c5955f08cae7","Type":"ContainerDied","Data":"f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154"}
Aug 04 15:44:38.645314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.645265    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr" event={"ID":"691d794d-cbd2-4751-be5a-c5955f08cae7","Type":"ContainerDied","Data":"7a6a0190778c4e6482801647c2a651f19fc950f51bca9b3c925e1131a2e1674f"}
Aug 04 15:44:38.653081 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.653064    2569 scope.go:117] "RemoveContainer" containerID="28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d"
Aug 04 15:44:38.653352 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:44:38.653330    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d\": container with ID starting with 28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d not found: ID does not exist" containerID="28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d"
Aug 04 15:44:38.653441 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.653358    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d"} err="failed to get container status \"28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d\": rpc error: code = NotFound desc = could not find container \"28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d\": container with ID starting with 28b59583261ec9ad63f2c1f55f0fa8289dc602bfe1ba8e9116e4b9bec4845a4d not found: ID does not exist"
Aug 04 15:44:38.653441 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.653376    2569 scope.go:117] "RemoveContainer" containerID="f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154"
Aug 04 15:44:38.661466 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.661450    2569 scope.go:117] "RemoveContainer" containerID="f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154"
Aug 04 15:44:38.661753 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:44:38.661733    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154\": container with ID starting with f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154 not found: ID does not exist" containerID="f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154"
Aug 04 15:44:38.661802 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.661761    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154"} err="failed to get container status \"f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154\": rpc error: code = NotFound desc = could not find container \"f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154\": container with ID starting with f7dc81e1f1e0dfad4f3a973d43e5c311d6fefc90142b25b7cc698ef35ee61154 not found: ID does not exist"
Aug 04 15:44:38.668187 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.668164    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"]
Aug 04 15:44:38.671201 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.671178    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-cde38272-76f7b58954-fbcdp"]
Aug 04 15:44:38.681960 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.681931    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"]
Aug 04 15:44:38.686759 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:38.686732    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-cde38272-995dfbd5b-d55kr"]
Aug 04 15:44:39.964720 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:39.964687    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b053fda-62c7-43d2-b527-9c6712de54cb" path="/var/lib/kubelet/pods/0b053fda-62c7-43d2-b527-9c6712de54cb/volumes"
Aug 04 15:44:39.965099 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:39.965055    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ded334d-2483-4e4d-a168-41a6ac0376e1" path="/var/lib/kubelet/pods/2ded334d-2483-4e4d-a168-41a6ac0376e1/volumes"
Aug 04 15:44:39.965362 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:39.965349    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="691d794d-cbd2-4751-be5a-c5955f08cae7" path="/var/lib/kubelet/pods/691d794d-cbd2-4751-be5a-c5955f08cae7/volumes"
Aug 04 15:44:40.920776 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:40.920738    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:44:53.918034 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:44:53.917996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:46:21.933930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:21.933885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:46:31.323140 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:31.323107    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:46:42.536162 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536123    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"]
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536450    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="691d794d-cbd2-4751-be5a-c5955f08cae7" containerName="payload-pre-processing"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536461    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="691d794d-cbd2-4751-be5a-c5955f08cae7" containerName="payload-pre-processing"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536469    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ded334d-2483-4e4d-a168-41a6ac0376e1" containerName="payload-processing"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536475    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ded334d-2483-4e4d-a168-41a6ac0376e1" containerName="payload-processing"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536485    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b053fda-62c7-43d2-b527-9c6712de54cb" containerName="revoke-keys"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536490    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b053fda-62c7-43d2-b527-9c6712de54cb" containerName="revoke-keys"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536543    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="691d794d-cbd2-4751-be5a-c5955f08cae7" containerName="payload-pre-processing"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536552    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ded334d-2483-4e4d-a168-41a6ac0376e1" containerName="payload-processing"
Aug 04 15:46:42.536604 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.536559    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b053fda-62c7-43d2-b527-9c6712de54cb" containerName="revoke-keys"
Aug 04 15:46:42.539607 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.539585    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.542930 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.542906    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ff74c09a\""
Aug 04 15:46:42.550317 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.550288    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"]
Aug 04 15:46:42.587465 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.587434    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"]
Aug 04 15:46:42.590599 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.590583    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.593226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.593202    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ff74c09a-dockercfg-bmdlr\""
Aug 04 15:46:42.594627 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.594596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9qr\" (UniqueName: \"kubernetes.io/projected/e827c84e-11ff-4527-822b-50fdca9b4a6e-kube-api-access-6m9qr\") pod \"payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.594764 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.594743    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e827c84e-11ff-4527-822b-50fdca9b4a6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.594830 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.594772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkgwm\" (UniqueName: \"kubernetes.io/projected/b4db94f0-9672-4d10-998c-512631677b1f-kube-api-access-bkgwm\") pod \"payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") " pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.594934 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.594834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4db94f0-9672-4d10-998c-512631677b1f-plugins-config-volume\") pod \"payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") " pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.601384 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.601357    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"]
Aug 04 15:46:42.696119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.696072    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9qr\" (UniqueName: \"kubernetes.io/projected/e827c84e-11ff-4527-822b-50fdca9b4a6e-kube-api-access-6m9qr\") pod \"payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.696338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.696144    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e827c84e-11ff-4527-822b-50fdca9b4a6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.696338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.696170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkgwm\" (UniqueName: \"kubernetes.io/projected/b4db94f0-9672-4d10-998c-512631677b1f-kube-api-access-bkgwm\") pod \"payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") " pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.696338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.696210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4db94f0-9672-4d10-998c-512631677b1f-plugins-config-volume\") pod \"payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") " pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.697207 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.697182    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4db94f0-9672-4d10-998c-512631677b1f-plugins-config-volume\") pod \"payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") " pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.697264 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.697191    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e827c84e-11ff-4527-822b-50fdca9b4a6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.704510 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.704478    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkgwm\" (UniqueName: \"kubernetes.io/projected/b4db94f0-9672-4d10-998c-512631677b1f-kube-api-access-bkgwm\") pod \"payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") " pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.704824 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.704806    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9qr\" (UniqueName: \"kubernetes.io/projected/e827c84e-11ff-4527-822b-50fdca9b4a6e-kube-api-access-6m9qr\") pod \"payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.851269 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.851168    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:42.903177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.902547    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:42.996036 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:42.995912    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"]
Aug 04 15:46:42.997951 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:46:42.997916    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode827c84e_11ff_4527_822b_50fdca9b4a6e.slice/crio-64074f2518badc73a50968b005ee26b348c6adaa8bf73f93f619f4396ed924f1 WatchSource:0}: Error finding container 64074f2518badc73a50968b005ee26b348c6adaa8bf73f93f619f4396ed924f1: Status 404 returned error can't find the container with id 64074f2518badc73a50968b005ee26b348c6adaa8bf73f93f619f4396ed924f1
Aug 04 15:46:43.052214 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.052187    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"]
Aug 04 15:46:43.054312 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:46:43.054285    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4db94f0_9672_4d10_998c_512631677b1f.slice/crio-7d014728c7de28bfe1d67cee30135fab3e1684164b8d6a0ef7af61885260a3e1 WatchSource:0}: Error finding container 7d014728c7de28bfe1d67cee30135fab3e1684164b8d6a0ef7af61885260a3e1: Status 404 returned error can't find the container with id 7d014728c7de28bfe1d67cee30135fab3e1684164b8d6a0ef7af61885260a3e1
Aug 04 15:46:43.105809 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:46:43.105774    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4db94f0_9672_4d10_998c_512631677b1f.slice/crio-6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11.scope\": RecentStats: unable to find data in memory cache]"
Aug 04 15:46:43.244356 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.244315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc" event={"ID":"b4db94f0-9672-4d10-998c-512631677b1f","Type":"ContainerStarted","Data":"6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11"}
Aug 04 15:46:43.244556 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.244517    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:43.244672 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.244560    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc" event={"ID":"b4db94f0-9672-4d10-998c-512631677b1f","Type":"ContainerStarted","Data":"7d014728c7de28bfe1d67cee30135fab3e1684164b8d6a0ef7af61885260a3e1"}
Aug 04 15:46:43.246104 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.246078    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq" event={"ID":"e827c84e-11ff-4527-822b-50fdca9b4a6e","Type":"ContainerStarted","Data":"05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8"}
Aug 04 15:46:43.246211 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.246126    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:46:43.246211 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.246146    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq" event={"ID":"e827c84e-11ff-4527-822b-50fdca9b4a6e","Type":"ContainerStarted","Data":"64074f2518badc73a50968b005ee26b348c6adaa8bf73f93f619f4396ed924f1"}
Aug 04 15:46:43.264614 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.264563    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc" podStartSLOduration=1.264544541 podStartE2EDuration="1.264544541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:46:43.262356809 +0000 UTC m=+2145.768654865" watchObservedRunningTime="2026-08-04 15:46:43.264544541 +0000 UTC m=+2145.770842596"
Aug 04 15:46:43.285415 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:43.285357    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq" podStartSLOduration=1.2853347290000001 podStartE2EDuration="1.285334729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:46:43.284407253 +0000 UTC m=+2145.790705310" watchObservedRunningTime="2026-08-04 15:46:43.285334729 +0000 UTC m=+2145.791632763"
Aug 04 15:46:44.618726 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:44.618688    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:46:54.253263 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:54.253177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:46:54.253918 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:46:54.253288    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:47:13.280268 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.280235    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"]
Aug 04 15:47:13.283676 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.283658    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:13.286718 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.286682    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:47:13.286883 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.286825    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:47:13.293499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.293472    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"]
Aug 04 15:47:13.381035 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.381003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef12586-459e-4d90-8848-d2e6c1a6550c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w\" (UID: \"eef12586-459e-4d90-8848-d2e6c1a6550c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:13.482340 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.482302    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef12586-459e-4d90-8848-d2e6c1a6550c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w\" (UID: \"eef12586-459e-4d90-8848-d2e6c1a6550c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:13.482976 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.482955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef12586-459e-4d90-8848-d2e6c1a6550c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w\" (UID: \"eef12586-459e-4d90-8848-d2e6c1a6550c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:13.595360 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.595254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:13.739193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:13.738993    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"]
Aug 04 15:47:13.742088 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:47:13.742054    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef12586_459e_4d90_8848_d2e6c1a6550c.slice/crio-8068381ba9fb21a278880ad3b32efe8809b86bbef7dbcfa2e6454b0eee2f8ed6 WatchSource:0}: Error finding container 8068381ba9fb21a278880ad3b32efe8809b86bbef7dbcfa2e6454b0eee2f8ed6: Status 404 returned error can't find the container with id 8068381ba9fb21a278880ad3b32efe8809b86bbef7dbcfa2e6454b0eee2f8ed6
Aug 04 15:47:14.389525 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:14.389469    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w" event={"ID":"eef12586-459e-4d90-8848-d2e6c1a6550c","Type":"ContainerStarted","Data":"60bec85720c48ca4bb6086fea6dae3af17516be60d480bfa0fead679b2d9930e"}
Aug 04 15:47:14.391947 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:14.389564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w" event={"ID":"eef12586-459e-4d90-8848-d2e6c1a6550c","Type":"ContainerStarted","Data":"8068381ba9fb21a278880ad3b32efe8809b86bbef7dbcfa2e6454b0eee2f8ed6"}
Aug 04 15:47:14.406675 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:14.406607    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w" podStartSLOduration=1.4065890460000001 podStartE2EDuration="1.406589046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:47:14.404736817 +0000 UTC m=+2176.911034875" watchObservedRunningTime="2026-08-04 15:47:14.406589046 +0000 UTC m=+2176.912887124"
Aug 04 15:47:15.395943 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:15.395903    2569 generic.go:356] "Generic (PLEG): container finished" podID="eef12586-459e-4d90-8848-d2e6c1a6550c" containerID="60bec85720c48ca4bb6086fea6dae3af17516be60d480bfa0fead679b2d9930e" exitCode=0
Aug 04 15:47:15.396340 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:15.395982    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w" event={"ID":"eef12586-459e-4d90-8848-d2e6c1a6550c","Type":"ContainerDied","Data":"60bec85720c48ca4bb6086fea6dae3af17516be60d480bfa0fead679b2d9930e"}
Aug 04 15:47:16.534324 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:16.534294    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:16.713078 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:16.712980    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef12586-459e-4d90-8848-d2e6c1a6550c-maas-api-service-ca\") pod \"eef12586-459e-4d90-8848-d2e6c1a6550c\" (UID: \"eef12586-459e-4d90-8848-d2e6c1a6550c\") "
Aug 04 15:47:16.713351 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:16.713326    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eef12586-459e-4d90-8848-d2e6c1a6550c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eef12586-459e-4d90-8848-d2e6c1a6550c" (UID: "eef12586-459e-4d90-8848-d2e6c1a6550c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:47:16.813805 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:16.813767    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eef12586-459e-4d90-8848-d2e6c1a6550c-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:47:17.407610 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:17.407572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w" event={"ID":"eef12586-459e-4d90-8848-d2e6c1a6550c","Type":"ContainerDied","Data":"8068381ba9fb21a278880ad3b32efe8809b86bbef7dbcfa2e6454b0eee2f8ed6"}
Aug 04 15:47:17.407828 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:17.407688    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8068381ba9fb21a278880ad3b32efe8809b86bbef7dbcfa2e6454b0eee2f8ed6"
Aug 04 15:47:17.407828 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:17.407704    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"
Aug 04 15:47:23.926260 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:23.926206    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"]
Aug 04 15:47:23.926756 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:23.926514    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc" podUID="b4db94f0-9672-4d10-998c-512631677b1f" containerName="payload-processing" containerID="cri-o://6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11" gracePeriod=30
Aug 04 15:47:23.936556 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:23.936524    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"]
Aug 04 15:47:23.936903 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:23.936836    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq" podUID="e827c84e-11ff-4527-822b-50fdca9b4a6e" containerName="payload-pre-processing" containerID="cri-o://05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8" gracePeriod=30
Aug 04 15:47:24.075109 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.075065    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"]
Aug 04 15:47:24.081057 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.081012    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ff74c09a-txx6w"]
Aug 04 15:47:24.194135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.194104    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:47:24.220071 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.220043    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:47:24.386399 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.386356    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e827c84e-11ff-4527-822b-50fdca9b4a6e-plugins-config-volume\") pod \"e827c84e-11ff-4527-822b-50fdca9b4a6e\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") "
Aug 04 15:47:24.386670 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.386411    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m9qr\" (UniqueName: \"kubernetes.io/projected/e827c84e-11ff-4527-822b-50fdca9b4a6e-kube-api-access-6m9qr\") pod \"e827c84e-11ff-4527-822b-50fdca9b4a6e\" (UID: \"e827c84e-11ff-4527-822b-50fdca9b4a6e\") "
Aug 04 15:47:24.386670 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.386458    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4db94f0-9672-4d10-998c-512631677b1f-plugins-config-volume\") pod \"b4db94f0-9672-4d10-998c-512631677b1f\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") "
Aug 04 15:47:24.386670 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.386552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkgwm\" (UniqueName: \"kubernetes.io/projected/b4db94f0-9672-4d10-998c-512631677b1f-kube-api-access-bkgwm\") pod \"b4db94f0-9672-4d10-998c-512631677b1f\" (UID: \"b4db94f0-9672-4d10-998c-512631677b1f\") "
Aug 04 15:47:24.386851 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.386830    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4db94f0-9672-4d10-998c-512631677b1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4db94f0-9672-4d10-998c-512631677b1f" (UID: "b4db94f0-9672-4d10-998c-512631677b1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:47:24.386904 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.386851    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e827c84e-11ff-4527-822b-50fdca9b4a6e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e827c84e-11ff-4527-822b-50fdca9b4a6e" (UID: "e827c84e-11ff-4527-822b-50fdca9b4a6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:47:24.388595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.388574    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4db94f0-9672-4d10-998c-512631677b1f-kube-api-access-bkgwm" (OuterVolumeSpecName: "kube-api-access-bkgwm") pod "b4db94f0-9672-4d10-998c-512631677b1f" (UID: "b4db94f0-9672-4d10-998c-512631677b1f"). InnerVolumeSpecName "kube-api-access-bkgwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:47:24.388663 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.388582    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e827c84e-11ff-4527-822b-50fdca9b4a6e-kube-api-access-6m9qr" (OuterVolumeSpecName: "kube-api-access-6m9qr") pod "e827c84e-11ff-4527-822b-50fdca9b4a6e" (UID: "e827c84e-11ff-4527-822b-50fdca9b4a6e"). InnerVolumeSpecName "kube-api-access-6m9qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:47:24.440936 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.440852    2569 generic.go:356] "Generic (PLEG): container finished" podID="e827c84e-11ff-4527-822b-50fdca9b4a6e" containerID="05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8" exitCode=0
Aug 04 15:47:24.440936 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.440907    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"
Aug 04 15:47:24.441163 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.440939    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq" event={"ID":"e827c84e-11ff-4527-822b-50fdca9b4a6e","Type":"ContainerDied","Data":"05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8"}
Aug 04 15:47:24.441163 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.440992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq" event={"ID":"e827c84e-11ff-4527-822b-50fdca9b4a6e","Type":"ContainerDied","Data":"64074f2518badc73a50968b005ee26b348c6adaa8bf73f93f619f4396ed924f1"}
Aug 04 15:47:24.441163 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.441016    2569 scope.go:117] "RemoveContainer" containerID="05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8"
Aug 04 15:47:24.442561 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.442541    2569 generic.go:356] "Generic (PLEG): container finished" podID="b4db94f0-9672-4d10-998c-512631677b1f" containerID="6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11" exitCode=0
Aug 04 15:47:24.442679 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.442580    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"
Aug 04 15:47:24.442679 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.442594    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc" event={"ID":"b4db94f0-9672-4d10-998c-512631677b1f","Type":"ContainerDied","Data":"6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11"}
Aug 04 15:47:24.442679 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.442627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc" event={"ID":"b4db94f0-9672-4d10-998c-512631677b1f","Type":"ContainerDied","Data":"7d014728c7de28bfe1d67cee30135fab3e1684164b8d6a0ef7af61885260a3e1"}
Aug 04 15:47:24.451001 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.450941    2569 scope.go:117] "RemoveContainer" containerID="05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8"
Aug 04 15:47:24.451350 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:47:24.451317    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8\": container with ID starting with 05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8 not found: ID does not exist" containerID="05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8"
Aug 04 15:47:24.451446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.451359    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8"} err="failed to get container status \"05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8\": rpc error: code = NotFound desc = could not find container \"05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8\": container with ID starting with 05327260b383fa926b0b6f481e7c9bab7e5a340a8babb6c9291be563345539e8 not found: ID does not exist"
Aug 04 15:47:24.451446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.451378    2569 scope.go:117] "RemoveContainer" containerID="6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11"
Aug 04 15:47:24.460085 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.460069    2569 scope.go:117] "RemoveContainer" containerID="6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11"
Aug 04 15:47:24.460348 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:47:24.460332    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11\": container with ID starting with 6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11 not found: ID does not exist" containerID="6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11"
Aug 04 15:47:24.460406 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.460355    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11"} err="failed to get container status \"6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11\": rpc error: code = NotFound desc = could not find container \"6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11\": container with ID starting with 6485e9c143c862820f99db6078f7bada6d03ac30de1097ad338c1abf01dbdf11 not found: ID does not exist"
Aug 04 15:47:24.487575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.487539    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e827c84e-11ff-4527-822b-50fdca9b4a6e-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:47:24.487575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.487570    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m9qr\" (UniqueName: \"kubernetes.io/projected/e827c84e-11ff-4527-822b-50fdca9b4a6e-kube-api-access-6m9qr\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:47:24.487575 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.487581    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4db94f0-9672-4d10-998c-512631677b1f-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:47:24.487890 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.487591    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkgwm\" (UniqueName: \"kubernetes.io/projected/b4db94f0-9672-4d10-998c-512631677b1f-kube-api-access-bkgwm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:47:24.489139 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.489113    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"]
Aug 04 15:47:24.495265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.495238    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ff74c09a-68498fb764-dw7wq"]
Aug 04 15:47:24.513800 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.513773    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"]
Aug 04 15:47:24.522016 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:24.521983    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ff74c09a-7c955ff44b-nlllc"]
Aug 04 15:47:25.964961 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:25.964923    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4db94f0-9672-4d10-998c-512631677b1f" path="/var/lib/kubelet/pods/b4db94f0-9672-4d10-998c-512631677b1f/volumes"
Aug 04 15:47:25.965483 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:25.965332    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e827c84e-11ff-4527-822b-50fdca9b4a6e" path="/var/lib/kubelet/pods/e827c84e-11ff-4527-822b-50fdca9b4a6e/volumes"
Aug 04 15:47:25.965701 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:25.965686    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef12586-459e-4d90-8848-d2e6c1a6550c" path="/var/lib/kubelet/pods/eef12586-459e-4d90-8848-d2e6c1a6550c/volumes"
Aug 04 15:47:26.015287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:47:26.015250    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:48:03.114022 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:48:03.113973    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:48:58.096158 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:48:58.096113    2569 scope.go:117] "RemoveContainer" containerID="4fb9b69862a6379d81b86419f5d80e22bf6e80c144b15d1492c1bdd39b2ccdb1"
Aug 04 15:49:49.414119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:49.414083    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:49:51.557312 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557273    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"]
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557624    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4db94f0-9672-4d10-998c-512631677b1f" containerName="payload-processing"
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557650    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4db94f0-9672-4d10-998c-512631677b1f" containerName="payload-processing"
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557662    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e827c84e-11ff-4527-822b-50fdca9b4a6e" containerName="payload-pre-processing"
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557667    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e827c84e-11ff-4527-822b-50fdca9b4a6e" containerName="payload-pre-processing"
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557677    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef12586-459e-4d90-8848-d2e6c1a6550c" containerName="revoke-keys"
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557683    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef12586-459e-4d90-8848-d2e6c1a6550c" containerName="revoke-keys"
Aug 04 15:49:51.557749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557745    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e827c84e-11ff-4527-822b-50fdca9b4a6e" containerName="payload-pre-processing"
Aug 04 15:49:51.557980 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557756    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef12586-459e-4d90-8848-d2e6c1a6550c" containerName="revoke-keys"
Aug 04 15:49:51.557980 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.557763    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4db94f0-9672-4d10-998c-512631677b1f" containerName="payload-processing"
Aug 04 15:49:51.560872 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.560848    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.564265 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.564236    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e4d56b38\""
Aug 04 15:49:51.573130 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.573098    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"]
Aug 04 15:49:51.576571 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.576541    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.577356 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.577328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"]
Aug 04 15:49:51.579427 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.579405    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e4d56b38-dockercfg-9hfkj\""
Aug 04 15:49:51.592019 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.591985    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"]
Aug 04 15:49:51.712148 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.712104    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6e1c90-b368-4d72-a8bd-3a455e7db360-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.712352 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.712156    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vbdl\" (UniqueName: \"kubernetes.io/projected/ca6e1c90-b368-4d72-a8bd-3a455e7db360-kube-api-access-2vbdl\") pod \"payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.712352 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.712239    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n822z\" (UniqueName: \"kubernetes.io/projected/b22994d7-7c0c-458c-82fe-27bb66a0cc92-kube-api-access-n822z\") pod \"payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") " pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.712352 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.712275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b22994d7-7c0c-458c-82fe-27bb66a0cc92-plugins-config-volume\") pod \"payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") " pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.813337 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.813233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6e1c90-b368-4d72-a8bd-3a455e7db360-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.813337 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.813276    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vbdl\" (UniqueName: \"kubernetes.io/projected/ca6e1c90-b368-4d72-a8bd-3a455e7db360-kube-api-access-2vbdl\") pod \"payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.813337 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.813310    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n822z\" (UniqueName: \"kubernetes.io/projected/b22994d7-7c0c-458c-82fe-27bb66a0cc92-kube-api-access-n822z\") pod \"payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") " pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.813337 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.813330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b22994d7-7c0c-458c-82fe-27bb66a0cc92-plugins-config-volume\") pod \"payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") " pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.814106 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.814079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b22994d7-7c0c-458c-82fe-27bb66a0cc92-plugins-config-volume\") pod \"payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") " pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.814211 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.814118    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6e1c90-b368-4d72-a8bd-3a455e7db360-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.822028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.821989    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n822z\" (UniqueName: \"kubernetes.io/projected/b22994d7-7c0c-458c-82fe-27bb66a0cc92-kube-api-access-n822z\") pod \"payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") " pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.822277 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.822257    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vbdl\" (UniqueName: \"kubernetes.io/projected/ca6e1c90-b368-4d72-a8bd-3a455e7db360-kube-api-access-2vbdl\") pod \"payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:51.871218 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.871177    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:51.890596 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:51.890560    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:52.020461 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.020432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"]
Aug 04 15:49:52.023889 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:49:52.023854    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22994d7_7c0c_458c_82fe_27bb66a0cc92.slice/crio-31afba909892cd8b25561a66ff5ba273f370f1b73b27ec5ef117a1bf4350013c WatchSource:0}: Error finding container 31afba909892cd8b25561a66ff5ba273f370f1b73b27ec5ef117a1bf4350013c: Status 404 returned error can't find the container with id 31afba909892cd8b25561a66ff5ba273f370f1b73b27ec5ef117a1bf4350013c
Aug 04 15:49:52.049433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.049405    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"]
Aug 04 15:49:52.060928 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:49:52.060897    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6e1c90_b368_4d72_a8bd_3a455e7db360.slice/crio-93f1b6fb595599db86b5522df76ee0999d2bf36ba0c84846e84ccd0288a235bb WatchSource:0}: Error finding container 93f1b6fb595599db86b5522df76ee0999d2bf36ba0c84846e84ccd0288a235bb: Status 404 returned error can't find the container with id 93f1b6fb595599db86b5522df76ee0999d2bf36ba0c84846e84ccd0288a235bb
Aug 04 15:49:52.134790 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.134753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s" event={"ID":"ca6e1c90-b368-4d72-a8bd-3a455e7db360","Type":"ContainerStarted","Data":"93f1b6fb595599db86b5522df76ee0999d2bf36ba0c84846e84ccd0288a235bb"}
Aug 04 15:49:52.136369 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.136334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq" event={"ID":"b22994d7-7c0c-458c-82fe-27bb66a0cc92","Type":"ContainerStarted","Data":"6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0"}
Aug 04 15:49:52.136512 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.136393    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq" event={"ID":"b22994d7-7c0c-458c-82fe-27bb66a0cc92","Type":"ContainerStarted","Data":"31afba909892cd8b25561a66ff5ba273f370f1b73b27ec5ef117a1bf4350013c"}
Aug 04 15:49:52.136512 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.136437    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:49:52.154542 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:52.154476    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq" podStartSLOduration=1.154451647 podStartE2EDuration="1.154451647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:49:52.152908322 +0000 UTC m=+2334.659206381" watchObservedRunningTime="2026-08-04 15:49:52.154451647 +0000 UTC m=+2334.660749705"
Aug 04 15:49:53.143760 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:53.143720    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s" event={"ID":"ca6e1c90-b368-4d72-a8bd-3a455e7db360","Type":"ContainerStarted","Data":"ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331"}
Aug 04 15:49:53.144214 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:53.144042    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:49:53.161085 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:53.161023    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s" podStartSLOduration=2.161002776 podStartE2EDuration="2.161002776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:49:53.160663288 +0000 UTC m=+2335.666961339" watchObservedRunningTime="2026-08-04 15:49:53.161002776 +0000 UTC m=+2335.667300824"
Aug 04 15:49:57.015361 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:57.015322    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:49:58.120436 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:58.120400    2569 scope.go:117] "RemoveContainer" containerID="c3d87ffdb5eab8a8e822e996cd6b638c8961c9e4569df367d378859844a93730"
Aug 04 15:49:58.129793 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:49:58.129767    2569 scope.go:117] "RemoveContainer" containerID="0583a93f0306f735ce700c550c8b970334d30ffa7cbf49f33414458678754731"
Aug 04 15:50:03.146205 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:03.146173    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:50:04.151291 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:04.151257    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:50:25.885886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.885848    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"]
Aug 04 15:50:25.889631 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.889606    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:25.892222 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.892197    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e4d56b38\"/\"openshift-service-ca.crt\""
Aug 04 15:50:25.893366 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.893339    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e4d56b38\"/\"kube-root-ca.crt\""
Aug 04 15:50:25.893506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.893339    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e4d56b38\"/\"default-dockercfg-shrpz\""
Aug 04 15:50:25.893506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.893409    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e4d56b38\"/\"e2e-lifecycle-model-e4d56b38-kserve-self-signed-certs\""
Aug 04 15:50:25.901707 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:25.901676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"]
Aug 04 15:50:26.015055 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.015013    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-dshm\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.015274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.015070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fefd68-9005-4e17-bd7e-26175e608e0f-tls-certs\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.015274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.015099    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-model-cache\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.015274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.015198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-home\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.015274 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.015233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-tmp-dir\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.015465 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.015282    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czfc6\" (UniqueName: \"kubernetes.io/projected/95fefd68-9005-4e17-bd7e-26175e608e0f-kube-api-access-czfc6\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.116797 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.116756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-dshm\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.116987 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.116918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fefd68-9005-4e17-bd7e-26175e608e0f-tls-certs\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.116987 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.116956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-model-cache\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.117096 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.117069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-home\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.117153 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.117138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-tmp-dir\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.117616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.117243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czfc6\" (UniqueName: \"kubernetes.io/projected/95fefd68-9005-4e17-bd7e-26175e608e0f-kube-api-access-czfc6\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.117616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.117444    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-home\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.117616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.117511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-tmp-dir\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.117876 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.117618    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-model-cache\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.119137 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.119116    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-dshm\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.119432 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.119412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fefd68-9005-4e17-bd7e-26175e608e0f-tls-certs\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.125740 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.125708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czfc6\" (UniqueName: \"kubernetes.io/projected/95fefd68-9005-4e17-bd7e-26175e608e0f-kube-api-access-czfc6\") pod \"e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") " pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.199347 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.199238    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:26.346109 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:26.343497    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"]
Aug 04 15:50:26.346873 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:50:26.346840    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95fefd68_9005_4e17_bd7e_26175e608e0f.slice/crio-dc10c9e9368364cd077dd91d59e2f08939f40d06e7112de8e3cad118bb2e927a WatchSource:0}: Error finding container dc10c9e9368364cd077dd91d59e2f08939f40d06e7112de8e3cad118bb2e927a: Status 404 returned error can't find the container with id dc10c9e9368364cd077dd91d59e2f08939f40d06e7112de8e3cad118bb2e927a
Aug 04 15:50:27.304552 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:27.304519    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh" event={"ID":"95fefd68-9005-4e17-bd7e-26175e608e0f","Type":"ContainerStarted","Data":"54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16"}
Aug 04 15:50:27.304552 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:27.304558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh" event={"ID":"95fefd68-9005-4e17-bd7e-26175e608e0f","Type":"ContainerStarted","Data":"dc10c9e9368364cd077dd91d59e2f08939f40d06e7112de8e3cad118bb2e927a"}
Aug 04 15:50:28.115380 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:28.115329    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh" podStartSLOduration=3.115309657 podStartE2EDuration="3.115309657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:50:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:50:27.323945305 +0000 UTC m=+2369.830243364" watchObservedRunningTime="2026-08-04 15:50:28.115309657 +0000 UTC m=+2370.621607760"
Aug 04 15:50:28.115599 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:28.115464    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:50:36.203258 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:36.203206    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:36.203770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:36.203676    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:36.216630 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:36.216599    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:36.357284 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:36.357243    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:50:43.696627 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:43.696586    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:50:58.153948 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:50:58.153915    2569 scope.go:117] "RemoveContainer" containerID="5c817bac5bce2c465f7fe6163a2c5b86da538308f53f8540f76280dea57f349c"
Aug 04 15:51:07.321704 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:07.321663    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:51:09.832715 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:09.832674    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"]
Aug 04 15:51:09.833125 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:09.833003    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq" podUID="b22994d7-7c0c-458c-82fe-27bb66a0cc92" containerName="payload-processing" containerID="cri-o://6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0" gracePeriod=30
Aug 04 15:51:09.840839 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:09.840813    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"]
Aug 04 15:51:09.841213 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:09.841088    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s" podUID="ca6e1c90-b368-4d72-a8bd-3a455e7db360" containerName="payload-pre-processing" containerID="cri-o://ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331" gracePeriod=30
Aug 04 15:51:10.108934 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.108909    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:51:10.115839 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.115811    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:51:10.219410 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.219354    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6e1c90-b368-4d72-a8bd-3a455e7db360-plugins-config-volume\") pod \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") "
Aug 04 15:51:10.219598 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.219418    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vbdl\" (UniqueName: \"kubernetes.io/projected/ca6e1c90-b368-4d72-a8bd-3a455e7db360-kube-api-access-2vbdl\") pod \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\" (UID: \"ca6e1c90-b368-4d72-a8bd-3a455e7db360\") "
Aug 04 15:51:10.219598 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.219522    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n822z\" (UniqueName: \"kubernetes.io/projected/b22994d7-7c0c-458c-82fe-27bb66a0cc92-kube-api-access-n822z\") pod \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") "
Aug 04 15:51:10.219747 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.219601    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b22994d7-7c0c-458c-82fe-27bb66a0cc92-plugins-config-volume\") pod \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\" (UID: \"b22994d7-7c0c-458c-82fe-27bb66a0cc92\") "
Aug 04 15:51:10.219814 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.219771    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca6e1c90-b368-4d72-a8bd-3a455e7db360-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca6e1c90-b368-4d72-a8bd-3a455e7db360" (UID: "ca6e1c90-b368-4d72-a8bd-3a455e7db360"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:51:10.219906 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.219883    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca6e1c90-b368-4d72-a8bd-3a455e7db360-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:10.220030 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.220008    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b22994d7-7c0c-458c-82fe-27bb66a0cc92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b22994d7-7c0c-458c-82fe-27bb66a0cc92" (UID: "b22994d7-7c0c-458c-82fe-27bb66a0cc92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:51:10.221630 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.221612    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b22994d7-7c0c-458c-82fe-27bb66a0cc92-kube-api-access-n822z" (OuterVolumeSpecName: "kube-api-access-n822z") pod "b22994d7-7c0c-458c-82fe-27bb66a0cc92" (UID: "b22994d7-7c0c-458c-82fe-27bb66a0cc92"). InnerVolumeSpecName "kube-api-access-n822z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:51:10.221721 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.221679    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca6e1c90-b368-4d72-a8bd-3a455e7db360-kube-api-access-2vbdl" (OuterVolumeSpecName: "kube-api-access-2vbdl") pod "ca6e1c90-b368-4d72-a8bd-3a455e7db360" (UID: "ca6e1c90-b368-4d72-a8bd-3a455e7db360"). InnerVolumeSpecName "kube-api-access-2vbdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:51:10.321239 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.321204    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n822z\" (UniqueName: \"kubernetes.io/projected/b22994d7-7c0c-458c-82fe-27bb66a0cc92-kube-api-access-n822z\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:10.321239 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.321232    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b22994d7-7c0c-458c-82fe-27bb66a0cc92-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:10.321239 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.321242    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vbdl\" (UniqueName: \"kubernetes.io/projected/ca6e1c90-b368-4d72-a8bd-3a455e7db360-kube-api-access-2vbdl\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:10.502731 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.502692    2569 generic.go:356] "Generic (PLEG): container finished" podID="ca6e1c90-b368-4d72-a8bd-3a455e7db360" containerID="ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331" exitCode=0
Aug 04 15:51:10.502932 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.502737    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"
Aug 04 15:51:10.502932 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.502789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s" event={"ID":"ca6e1c90-b368-4d72-a8bd-3a455e7db360","Type":"ContainerDied","Data":"ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331"}
Aug 04 15:51:10.502932 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.502849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s" event={"ID":"ca6e1c90-b368-4d72-a8bd-3a455e7db360","Type":"ContainerDied","Data":"93f1b6fb595599db86b5522df76ee0999d2bf36ba0c84846e84ccd0288a235bb"}
Aug 04 15:51:10.502932 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.502874    2569 scope.go:117] "RemoveContainer" containerID="ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331"
Aug 04 15:51:10.504343 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.504313    2569 generic.go:356] "Generic (PLEG): container finished" podID="b22994d7-7c0c-458c-82fe-27bb66a0cc92" containerID="6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0" exitCode=0
Aug 04 15:51:10.504450 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.504350    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"
Aug 04 15:51:10.504450 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.504354    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq" event={"ID":"b22994d7-7c0c-458c-82fe-27bb66a0cc92","Type":"ContainerDied","Data":"6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0"}
Aug 04 15:51:10.504564 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.504459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq" event={"ID":"b22994d7-7c0c-458c-82fe-27bb66a0cc92","Type":"ContainerDied","Data":"31afba909892cd8b25561a66ff5ba273f370f1b73b27ec5ef117a1bf4350013c"}
Aug 04 15:51:10.511933 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.511906    2569 scope.go:117] "RemoveContainer" containerID="ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331"
Aug 04 15:51:10.512354 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:51:10.512211    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331\": container with ID starting with ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331 not found: ID does not exist" containerID="ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331"
Aug 04 15:51:10.512354 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.512240    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331"} err="failed to get container status \"ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331\": rpc error: code = NotFound desc = could not find container \"ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331\": container with ID starting with ebfaa3598a742a6e62ed2f4b47e056b7bc7d9708f8f133cb8a7d4aa0951bf331 not found: ID does not exist"
Aug 04 15:51:10.512354 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.512263    2569 scope.go:117] "RemoveContainer" containerID="6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0"
Aug 04 15:51:10.522119 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.521758    2569 scope.go:117] "RemoveContainer" containerID="6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0"
Aug 04 15:51:10.522336 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:51:10.522296    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0\": container with ID starting with 6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0 not found: ID does not exist" containerID="6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0"
Aug 04 15:51:10.522472 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.522334    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0"} err="failed to get container status \"6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0\": rpc error: code = NotFound desc = could not find container \"6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0\": container with ID starting with 6064454a798520078c62e6bc706b11dadd7d90141b491331faeee7df9352b7b0 not found: ID does not exist"
Aug 04 15:51:10.530226 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.530193    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"]
Aug 04 15:51:10.534660 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.534619    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e4d56b38-55c86c6cb4-q2n4s"]
Aug 04 15:51:10.546531 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.546505    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"]
Aug 04 15:51:10.550532 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:10.550504    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e4d56b38-5fb4db7f94-qxchq"]
Aug 04 15:51:11.964607 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:11.964566    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b22994d7-7c0c-458c-82fe-27bb66a0cc92" path="/var/lib/kubelet/pods/b22994d7-7c0c-458c-82fe-27bb66a0cc92/volumes"
Aug 04 15:51:11.965028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:11.964992    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca6e1c90-b368-4d72-a8bd-3a455e7db360" path="/var/lib/kubelet/pods/ca6e1c90-b368-4d72-a8bd-3a455e7db360/volumes"
Aug 04 15:51:13.217945 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:13.217904    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:51:18.107101 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:18.107061    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"]
Aug 04 15:51:33.126584 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.126455    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh" podUID="95fefd68-9005-4e17-bd7e-26175e608e0f" containerName="main" containerID="cri-o://54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16" gracePeriod=45
Aug 04 15:51:33.383995 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.383922    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:51:33.532734 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.532685    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-home\") pod \"95fefd68-9005-4e17-bd7e-26175e608e0f\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") "
Aug 04 15:51:33.532734 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.532737    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-tmp-dir\") pod \"95fefd68-9005-4e17-bd7e-26175e608e0f\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") "
Aug 04 15:51:33.532734 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.532759    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fefd68-9005-4e17-bd7e-26175e608e0f-tls-certs\") pod \"95fefd68-9005-4e17-bd7e-26175e608e0f\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") "
Aug 04 15:51:33.533086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.532775    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-model-cache\") pod \"95fefd68-9005-4e17-bd7e-26175e608e0f\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") "
Aug 04 15:51:33.533086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.532816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czfc6\" (UniqueName: \"kubernetes.io/projected/95fefd68-9005-4e17-bd7e-26175e608e0f-kube-api-access-czfc6\") pod \"95fefd68-9005-4e17-bd7e-26175e608e0f\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") "
Aug 04 15:51:33.533086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.532901    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-dshm\") pod \"95fefd68-9005-4e17-bd7e-26175e608e0f\" (UID: \"95fefd68-9005-4e17-bd7e-26175e608e0f\") "
Aug 04 15:51:33.533086 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.533047    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-home" (OuterVolumeSpecName: "home") pod "95fefd68-9005-4e17-bd7e-26175e608e0f" (UID: "95fefd68-9005-4e17-bd7e-26175e608e0f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:51:33.533262 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.533041    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95fefd68-9005-4e17-bd7e-26175e608e0f" (UID: "95fefd68-9005-4e17-bd7e-26175e608e0f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:51:33.533262 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.533118    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-model-cache" (OuterVolumeSpecName: "model-cache") pod "95fefd68-9005-4e17-bd7e-26175e608e0f" (UID: "95fefd68-9005-4e17-bd7e-26175e608e0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:51:33.535163 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.535135    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95fefd68-9005-4e17-bd7e-26175e608e0f-kube-api-access-czfc6" (OuterVolumeSpecName: "kube-api-access-czfc6") pod "95fefd68-9005-4e17-bd7e-26175e608e0f" (UID: "95fefd68-9005-4e17-bd7e-26175e608e0f"). InnerVolumeSpecName "kube-api-access-czfc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:51:33.535255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.535186    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-dshm" (OuterVolumeSpecName: "dshm") pod "95fefd68-9005-4e17-bd7e-26175e608e0f" (UID: "95fefd68-9005-4e17-bd7e-26175e608e0f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:51:33.535255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.535190    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95fefd68-9005-4e17-bd7e-26175e608e0f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95fefd68-9005-4e17-bd7e-26175e608e0f" (UID: "95fefd68-9005-4e17-bd7e-26175e608e0f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:51:33.609700 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.609633    2569 generic.go:356] "Generic (PLEG): container finished" podID="95fefd68-9005-4e17-bd7e-26175e608e0f" containerID="54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16" exitCode=0
Aug 04 15:51:33.609700 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.609705    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"
Aug 04 15:51:33.609923 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.609733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh" event={"ID":"95fefd68-9005-4e17-bd7e-26175e608e0f","Type":"ContainerDied","Data":"54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16"}
Aug 04 15:51:33.609923 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.609769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh" event={"ID":"95fefd68-9005-4e17-bd7e-26175e608e0f","Type":"ContainerDied","Data":"dc10c9e9368364cd077dd91d59e2f08939f40d06e7112de8e3cad118bb2e927a"}
Aug 04 15:51:33.609923 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.609787    2569 scope.go:117] "RemoveContainer" containerID="54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16"
Aug 04 15:51:33.618803 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.618782    2569 scope.go:117] "RemoveContainer" containerID="54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16"
Aug 04 15:51:33.619109 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:51:33.619087    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16\": container with ID starting with 54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16 not found: ID does not exist" containerID="54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16"
Aug 04 15:51:33.619204 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.619119    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16"} err="failed to get container status \"54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16\": rpc error: code = NotFound desc = could not find container \"54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16\": container with ID starting with 54041b3cbbec24686746c34d678b7e28c249f21c4644b43b5282ce3a2f9b6c16 not found: ID does not exist"
Aug 04 15:51:33.634198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.634076    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:33.634198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.634107    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:33.634198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.634156    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:33.634198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.634168    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95fefd68-9005-4e17-bd7e-26175e608e0f-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:33.634198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.634182    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95fefd68-9005-4e17-bd7e-26175e608e0f-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:33.634198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.634194    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czfc6\" (UniqueName: \"kubernetes.io/projected/95fefd68-9005-4e17-bd7e-26175e608e0f-kube-api-access-czfc6\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:51:33.635955 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.635930    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"]
Aug 04 15:51:33.641164 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.641136    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e4d56b38/e2e-lifecycle-model-e4d56b38-kserve-5f5c9b48ff-987jh"]
Aug 04 15:51:33.964582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:51:33.964493    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95fefd68-9005-4e17-bd7e-26175e608e0f" path="/var/lib/kubelet/pods/95fefd68-9005-4e17-bd7e-26175e608e0f/volumes"
Aug 04 15:52:27.029256 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:52:27.029213    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:52:38.527215 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:52:38.527180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:52:45.223486 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:52:45.223449    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:52:54.017820 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:52:54.017780    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:53:36.800082 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800036    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"]
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800514    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca6e1c90-b368-4d72-a8bd-3a455e7db360" containerName="payload-pre-processing"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800532    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca6e1c90-b368-4d72-a8bd-3a455e7db360" containerName="payload-pre-processing"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800547    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95fefd68-9005-4e17-bd7e-26175e608e0f" containerName="main"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800555    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="95fefd68-9005-4e17-bd7e-26175e608e0f" containerName="main"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800569    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b22994d7-7c0c-458c-82fe-27bb66a0cc92" containerName="payload-processing"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800579    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b22994d7-7c0c-458c-82fe-27bb66a0cc92" containerName="payload-processing"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800693    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b22994d7-7c0c-458c-82fe-27bb66a0cc92" containerName="payload-processing"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800707    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca6e1c90-b368-4d72-a8bd-3a455e7db360" containerName="payload-pre-processing"
Aug 04 15:53:36.800770 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.800721    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="95fefd68-9005-4e17-bd7e-26175e608e0f" containerName="main"
Aug 04 15:53:36.803763 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.803740    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:36.806514 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.806491    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-21cd04-dockercfg-xgprz\""
Aug 04 15:53:36.806690 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.806553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-21cd04\""
Aug 04 15:53:36.813176 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.813152    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"]
Aug 04 15:53:36.821860 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.821831    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"]
Aug 04 15:53:36.825159 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.825134    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:36.837480 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.837449    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"]
Aug 04 15:53:36.908410 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.908371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnw5g\" (UniqueName: \"kubernetes.io/projected/2bda31ed-f388-4260-b963-be76cb08fe5b-kube-api-access-pnw5g\") pod \"payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:36.908600 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.908424    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bda31ed-f388-4260-b963-be76cb08fe5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:36.908600 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.908451    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8p47\" (UniqueName: \"kubernetes.io/projected/fdc28f02-515b-40ff-83a1-a47af8b9378b-kube-api-access-k8p47\") pod \"payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:36.908600 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:36.908518    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc28f02-515b-40ff-83a1-a47af8b9378b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:37.009697 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.009629    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnw5g\" (UniqueName: \"kubernetes.io/projected/2bda31ed-f388-4260-b963-be76cb08fe5b-kube-api-access-pnw5g\") pod \"payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:37.009873 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.009710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bda31ed-f388-4260-b963-be76cb08fe5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:37.009916 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.009866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8p47\" (UniqueName: \"kubernetes.io/projected/fdc28f02-515b-40ff-83a1-a47af8b9378b-kube-api-access-k8p47\") pod \"payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:37.009979 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.009932    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc28f02-515b-40ff-83a1-a47af8b9378b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:37.010475 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.010455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bda31ed-f388-4260-b963-be76cb08fe5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:37.010746 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.010722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc28f02-515b-40ff-83a1-a47af8b9378b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:37.022168 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.022132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8p47\" (UniqueName: \"kubernetes.io/projected/fdc28f02-515b-40ff-83a1-a47af8b9378b-kube-api-access-k8p47\") pod \"payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:37.022358 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.022130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnw5g\" (UniqueName: \"kubernetes.io/projected/2bda31ed-f388-4260-b963-be76cb08fe5b-kube-api-access-pnw5g\") pod \"payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:37.115158 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.115119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:37.136618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.136583    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:37.270244 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.270214    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"]
Aug 04 15:53:37.273517 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:53:37.273482    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc28f02_515b_40ff_83a1_a47af8b9378b.slice/crio-965a92e8ee1f7be1fe60f8ddb26c4e9bb964a39973f6510b4b87db94dcfb9f22 WatchSource:0}: Error finding container 965a92e8ee1f7be1fe60f8ddb26c4e9bb964a39973f6510b4b87db94dcfb9f22: Status 404 returned error can't find the container with id 965a92e8ee1f7be1fe60f8ddb26c4e9bb964a39973f6510b4b87db94dcfb9f22
Aug 04 15:53:37.294168 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:37.294135    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"]
Aug 04 15:53:37.309745 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:53:37.309714    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bda31ed_f388_4260_b963_be76cb08fe5b.slice/crio-dcd2c741b84978dec39606656416ac6a5902fc49a990630540218d50ba2d44bd WatchSource:0}: Error finding container dcd2c741b84978dec39606656416ac6a5902fc49a990630540218d50ba2d44bd: Status 404 returned error can't find the container with id dcd2c741b84978dec39606656416ac6a5902fc49a990630540218d50ba2d44bd
Aug 04 15:53:38.165819 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.165773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms" event={"ID":"fdc28f02-515b-40ff-83a1-a47af8b9378b","Type":"ContainerStarted","Data":"554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665"}
Aug 04 15:53:38.166314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.165842    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms" event={"ID":"fdc28f02-515b-40ff-83a1-a47af8b9378b","Type":"ContainerStarted","Data":"965a92e8ee1f7be1fe60f8ddb26c4e9bb964a39973f6510b4b87db94dcfb9f22"}
Aug 04 15:53:38.166314 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.165874    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:38.167390 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.167364    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d" event={"ID":"2bda31ed-f388-4260-b963-be76cb08fe5b","Type":"ContainerStarted","Data":"80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da"}
Aug 04 15:53:38.167530 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.167402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d" event={"ID":"2bda31ed-f388-4260-b963-be76cb08fe5b","Type":"ContainerStarted","Data":"dcd2c741b84978dec39606656416ac6a5902fc49a990630540218d50ba2d44bd"}
Aug 04 15:53:38.167530 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.167480    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:38.187860 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.187783    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms" podStartSLOduration=2.187762171 podStartE2EDuration="2.187762171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:53:38.185222965 +0000 UTC m=+2560.691521023" watchObservedRunningTime="2026-08-04 15:53:38.187762171 +0000 UTC m=+2560.694060228"
Aug 04 15:53:38.206049 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.205982    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d" podStartSLOduration=2.205962562 podStartE2EDuration="2.205962562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:53:38.205138577 +0000 UTC m=+2560.711436645" watchObservedRunningTime="2026-08-04 15:53:38.205962562 +0000 UTC m=+2560.712260620"
Aug 04 15:53:38.722655 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:38.722606    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:53:49.174855 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:49.174822    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:53:49.175355 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:49.174896    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:53:58.197754 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:53:58.197722    2569 scope.go:117] "RemoveContainer" containerID="60bec85720c48ca4bb6086fea6dae3af17516be60d480bfa0fead679b2d9930e"
Aug 04 15:54:05.928203 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.928160    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"]
Aug 04 15:54:05.932891 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.932865    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:05.935498 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.935473    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-9d3b05\""
Aug 04 15:54:05.943002 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.942972    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"]
Aug 04 15:54:05.968001 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.967956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a267891b-04a0-42ff-a155-9f746c4deeac-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:05.968210 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.968040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tjnp\" (UniqueName: \"kubernetes.io/projected/a267891b-04a0-42ff-a155-9f746c4deeac-kube-api-access-7tjnp\") pod \"payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:05.978021 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.977989    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"]
Aug 04 15:54:05.981224 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.981203    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:05.984187 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.984153    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-9d3b05-dockercfg-bkvls\""
Aug 04 15:54:05.994777 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:05.994744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"]
Aug 04 15:54:06.068545 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.068491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2d5n\" (UniqueName: \"kubernetes.io/projected/4dc8045e-563b-4b20-984a-a20905c8fc48-kube-api-access-j2d5n\") pod \"payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.068772 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.068562    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a267891b-04a0-42ff-a155-9f746c4deeac-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:06.068772 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.068696    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tjnp\" (UniqueName: \"kubernetes.io/projected/a267891b-04a0-42ff-a155-9f746c4deeac-kube-api-access-7tjnp\") pod \"payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:06.068772 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.068735    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc8045e-563b-4b20-984a-a20905c8fc48-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.069317 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.069297    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a267891b-04a0-42ff-a155-9f746c4deeac-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:06.077594 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.077562    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tjnp\" (UniqueName: \"kubernetes.io/projected/a267891b-04a0-42ff-a155-9f746c4deeac-kube-api-access-7tjnp\") pod \"payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:06.169786 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.169740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2d5n\" (UniqueName: \"kubernetes.io/projected/4dc8045e-563b-4b20-984a-a20905c8fc48-kube-api-access-j2d5n\") pod \"payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.170001 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.169862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc8045e-563b-4b20-984a-a20905c8fc48-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.170578 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.170551    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc8045e-563b-4b20-984a-a20905c8fc48-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.179287 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.179214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2d5n\" (UniqueName: \"kubernetes.io/projected/4dc8045e-563b-4b20-984a-a20905c8fc48-kube-api-access-j2d5n\") pod \"payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.245541 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.245489    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:06.293734 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.293693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:06.402129 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.402094    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"]
Aug 04 15:54:06.403353 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:54:06.403311    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda267891b_04a0_42ff_a155_9f746c4deeac.slice/crio-85980d10be0f6e34ee280bf8ea55003679e5682610012caab89a1a4230bdd66d WatchSource:0}: Error finding container 85980d10be0f6e34ee280bf8ea55003679e5682610012caab89a1a4230bdd66d: Status 404 returned error can't find the container with id 85980d10be0f6e34ee280bf8ea55003679e5682610012caab89a1a4230bdd66d
Aug 04 15:54:06.456911 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:06.456886    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"]
Aug 04 15:54:06.466943 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:54:06.466909    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc8045e_563b_4b20_984a_a20905c8fc48.slice/crio-6147257fe6f168ac7b1976bb9700b1956950811c7be69e58ebe3005584aa6224 WatchSource:0}: Error finding container 6147257fe6f168ac7b1976bb9700b1956950811c7be69e58ebe3005584aa6224: Status 404 returned error can't find the container with id 6147257fe6f168ac7b1976bb9700b1956950811c7be69e58ebe3005584aa6224
Aug 04 15:54:07.300649 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.300593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" event={"ID":"4dc8045e-563b-4b20-984a-a20905c8fc48","Type":"ContainerStarted","Data":"ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f"}
Aug 04 15:54:07.301174 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.300765    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:07.301174 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.300800    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" event={"ID":"4dc8045e-563b-4b20-984a-a20905c8fc48","Type":"ContainerStarted","Data":"6147257fe6f168ac7b1976bb9700b1956950811c7be69e58ebe3005584aa6224"}
Aug 04 15:54:07.302078 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.302050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" event={"ID":"a267891b-04a0-42ff-a155-9f746c4deeac","Type":"ContainerStarted","Data":"fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756"}
Aug 04 15:54:07.302228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.302091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" event={"ID":"a267891b-04a0-42ff-a155-9f746c4deeac","Type":"ContainerStarted","Data":"85980d10be0f6e34ee280bf8ea55003679e5682610012caab89a1a4230bdd66d"}
Aug 04 15:54:07.302228 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.302184    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:07.321581 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.321516    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" podStartSLOduration=2.321477768 podStartE2EDuration="2.321477768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:54:07.320353738 +0000 UTC m=+2589.826651794" watchObservedRunningTime="2026-08-04 15:54:07.321477768 +0000 UTC m=+2589.827775824"
Aug 04 15:54:07.337881 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:07.337816    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" podStartSLOduration=2.337797011 podStartE2EDuration="2.337797011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:54:07.336443532 +0000 UTC m=+2589.842741618" watchObservedRunningTime="2026-08-04 15:54:07.337797011 +0000 UTC m=+2589.844095066"
Aug 04 15:54:08.727193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:08.727158    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:54:18.308526 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:18.308489    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:54:18.308994 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:18.308778    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:54:38.596330 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.596297    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"]
Aug 04 15:54:38.599991 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.599968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.602845 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.602824    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-21cd04\"/\"kube-root-ca.crt\""
Aug 04 15:54:38.603738 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.603722    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-21cd04\"/\"openshift-service-ca.crt\""
Aug 04 15:54:38.603819 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.603802    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-21cd04\"/\"test-model-21cd04-kserve-self-signed-certs\""
Aug 04 15:54:38.603882 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.603724    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-21cd04\"/\"default-dockercfg-6zkcs\""
Aug 04 15:54:38.610387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.610362    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"]
Aug 04 15:54:38.775913 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.775868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-dshm\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.776135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.775987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-home\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.776135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.776015    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2j4q\" (UniqueName: \"kubernetes.io/projected/8a231a83-d39f-4e80-8be5-17492f676f10-kube-api-access-s2j4q\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.776135 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.776072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a231a83-d39f-4e80-8be5-17492f676f10-tls-certs\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.776311 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.776142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-tmp-dir\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.776311 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.776200    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-model-cache\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877270 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-dshm\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877484 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-home\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877484 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2j4q\" (UniqueName: \"kubernetes.io/projected/8a231a83-d39f-4e80-8be5-17492f676f10-kube-api-access-s2j4q\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877484 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877424    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a231a83-d39f-4e80-8be5-17492f676f10-tls-certs\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877484 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-tmp-dir\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877753 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-model-cache\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877817 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877789    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-home\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877874 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877838    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-tmp-dir\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.877950 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.877927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-model-cache\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.879499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.879479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-dshm\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.879983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.879960    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a231a83-d39f-4e80-8be5-17492f676f10-tls-certs\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.887148 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.887122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2j4q\" (UniqueName: \"kubernetes.io/projected/8a231a83-d39f-4e80-8be5-17492f676f10-kube-api-access-s2j4q\") pod \"test-model-21cd04-kserve-6c5668b4fc-gccgb\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") " pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:38.910830 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:38.910788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:39.052967 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:39.052909    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"]
Aug 04 15:54:39.055427 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:54:39.055396    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a231a83_d39f_4e80_8be5_17492f676f10.slice/crio-07a393411cba973c690bc68cdd99b044336ab2fa574e9883dc2a6a2658b08a21 WatchSource:0}: Error finding container 07a393411cba973c690bc68cdd99b044336ab2fa574e9883dc2a6a2658b08a21: Status 404 returned error can't find the container with id 07a393411cba973c690bc68cdd99b044336ab2fa574e9883dc2a6a2658b08a21
Aug 04 15:54:39.451742 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:39.451695    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb" event={"ID":"8a231a83-d39f-4e80-8be5-17492f676f10","Type":"ContainerStarted","Data":"6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa"}
Aug 04 15:54:39.451742 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:39.451747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb" event={"ID":"8a231a83-d39f-4e80-8be5-17492f676f10","Type":"ContainerStarted","Data":"07a393411cba973c690bc68cdd99b044336ab2fa574e9883dc2a6a2658b08a21"}
Aug 04 15:54:39.473957 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:39.473904    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb" podStartSLOduration=1.473888828 podStartE2EDuration="1.473888828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:54:39.472561518 +0000 UTC m=+2621.978859576" watchObservedRunningTime="2026-08-04 15:54:39.473888828 +0000 UTC m=+2621.980186883"
Aug 04 15:54:41.914623 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:41.914586    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:54:48.911330 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:48.911279    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:48.911807 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:48.911345    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:48.924231 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:48.924199    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:49.509325 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:49.509289    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:54:58.454601 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.454567    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"]
Aug 04 15:54:58.458155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.458136    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.464324 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.464291    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9d3b05\"/\"default-dockercfg-7tbqh\""
Aug 04 15:54:58.464497 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.464291    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9d3b05\"/\"kube-root-ca.crt\""
Aug 04 15:54:58.464497 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.464337    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9d3b05\"/\"test-model-9d3b05-kserve-self-signed-certs\""
Aug 04 15:54:58.464957 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.464937    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9d3b05\"/\"openshift-service-ca.crt\""
Aug 04 15:54:58.473893 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.473859    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"]
Aug 04 15:54:58.577956 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.577919    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11df3234-77f8-4ef1-822b-53edd523e4e2-tls-certs\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.578147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.577976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-model-cache\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.578147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.578002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8jt\" (UniqueName: \"kubernetes.io/projected/11df3234-77f8-4ef1-822b-53edd523e4e2-kube-api-access-cg8jt\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.578147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.578063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-home\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.578147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.578101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-dshm\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.578147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.578134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-tmp-dir\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.679532 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.679495    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11df3234-77f8-4ef1-822b-53edd523e4e2-tls-certs\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.679766 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.679550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-model-cache\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.679766 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.679580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8jt\" (UniqueName: \"kubernetes.io/projected/11df3234-77f8-4ef1-822b-53edd523e4e2-kube-api-access-cg8jt\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.679766 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.679606    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-home\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.679766 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.679634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-dshm\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.679766 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.679698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-tmp-dir\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.680118 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.680084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-home\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.680243 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.680163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-model-cache\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.680243 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.680170    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-tmp-dir\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.681990 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.681965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-dshm\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.682236 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.682217    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11df3234-77f8-4ef1-822b-53edd523e4e2-tls-certs\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.689537 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.689498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8jt\" (UniqueName: \"kubernetes.io/projected/11df3234-77f8-4ef1-822b-53edd523e4e2-kube-api-access-cg8jt\") pod \"test-model-9d3b05-kserve-587475677f-djt9b\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") " pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.768415 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.768318    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:54:58.930060 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:58.930004    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"]
Aug 04 15:54:58.933969 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:54:58.933937    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11df3234_77f8_4ef1_822b_53edd523e4e2.slice/crio-901edf5a0506babfc449685ceae2c9dd1cfe8431f7a544202754f0527b8862e9 WatchSource:0}: Error finding container 901edf5a0506babfc449685ceae2c9dd1cfe8431f7a544202754f0527b8862e9: Status 404 returned error can't find the container with id 901edf5a0506babfc449685ceae2c9dd1cfe8431f7a544202754f0527b8862e9
Aug 04 15:54:59.551563 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:59.551530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b" event={"ID":"11df3234-77f8-4ef1-822b-53edd523e4e2","Type":"ContainerStarted","Data":"6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f"}
Aug 04 15:54:59.552001 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:59.551569    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b" event={"ID":"11df3234-77f8-4ef1-822b-53edd523e4e2","Type":"ContainerStarted","Data":"901edf5a0506babfc449685ceae2c9dd1cfe8431f7a544202754f0527b8862e9"}
Aug 04 15:54:59.574735 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:54:59.574678    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b" podStartSLOduration=1.5746619549999998 podStartE2EDuration="1.574661955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:54:59.573105683 +0000 UTC m=+2642.079403741" watchObservedRunningTime="2026-08-04 15:54:59.574661955 +0000 UTC m=+2642.080959989"
Aug 04 15:55:01.430318 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:01.430272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:55:08.768772 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:08.768737    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:55:08.769210 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:08.768794    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:55:08.781814 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:08.781787    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:55:09.606884 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:09.606855    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:55:10.116674 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:10.116624    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:55:38.792253 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:38.792215    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"]
Aug 04 15:55:39.703771 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.703739    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"]
Aug 04 15:55:39.833842 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.833805    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"]
Aug 04 15:55:39.837482 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.837461    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:39.839908 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.839886    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:55:39.840015 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.839943    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:55:39.845192 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.845160    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"]
Aug 04 15:55:39.963480 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:39.963381    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa04182b-7aeb-4269-96ea-b260e2d47b9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz\" (UID: \"fa04182b-7aeb-4269-96ea-b260e2d47b9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:40.064238 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.064200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa04182b-7aeb-4269-96ea-b260e2d47b9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz\" (UID: \"fa04182b-7aeb-4269-96ea-b260e2d47b9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:40.064903 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.064882    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa04182b-7aeb-4269-96ea-b260e2d47b9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz\" (UID: \"fa04182b-7aeb-4269-96ea-b260e2d47b9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:40.150847 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.150806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:40.290548 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.290514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"]
Aug 04 15:55:40.293188 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:55:40.293157    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa04182b_7aeb_4269_96ea_b260e2d47b9e.slice/crio-5bd1459707b746b0b6fd657722379ab62f875e4c15cad2b4b988fd2e03005462 WatchSource:0}: Error finding container 5bd1459707b746b0b6fd657722379ab62f875e4c15cad2b4b988fd2e03005462: Status 404 returned error can't find the container with id 5bd1459707b746b0b6fd657722379ab62f875e4c15cad2b4b988fd2e03005462
Aug 04 15:55:40.740042 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.740006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz" event={"ID":"fa04182b-7aeb-4269-96ea-b260e2d47b9e","Type":"ContainerStarted","Data":"9054b335c437b768d10e9a406456da460b0f03b6fee5257fca3e019a84e71f1b"}
Aug 04 15:55:40.740329 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.740062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz" event={"ID":"fa04182b-7aeb-4269-96ea-b260e2d47b9e","Type":"ContainerStarted","Data":"5bd1459707b746b0b6fd657722379ab62f875e4c15cad2b4b988fd2e03005462"}
Aug 04 15:55:40.757826 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:40.757767    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz" podStartSLOduration=1.757748808 podStartE2EDuration="1.757748808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:55:40.755731846 +0000 UTC m=+2683.262029901" watchObservedRunningTime="2026-08-04 15:55:40.757748808 +0000 UTC m=+2683.264046916"
Aug 04 15:55:41.746595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:41.746555    2569 generic.go:356] "Generic (PLEG): container finished" podID="fa04182b-7aeb-4269-96ea-b260e2d47b9e" containerID="9054b335c437b768d10e9a406456da460b0f03b6fee5257fca3e019a84e71f1b" exitCode=0
Aug 04 15:55:41.747032 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:41.746659    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz" event={"ID":"fa04182b-7aeb-4269-96ea-b260e2d47b9e","Type":"ContainerDied","Data":"9054b335c437b768d10e9a406456da460b0f03b6fee5257fca3e019a84e71f1b"}
Aug 04 15:55:42.315440 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:42.315402    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:55:42.886946 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:42.886920    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:43.001006 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:43.000962    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa04182b-7aeb-4269-96ea-b260e2d47b9e-maas-api-service-ca\") pod \"fa04182b-7aeb-4269-96ea-b260e2d47b9e\" (UID: \"fa04182b-7aeb-4269-96ea-b260e2d47b9e\") "
Aug 04 15:55:43.001378 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:43.001352    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa04182b-7aeb-4269-96ea-b260e2d47b9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa04182b-7aeb-4269-96ea-b260e2d47b9e" (UID: "fa04182b-7aeb-4269-96ea-b260e2d47b9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:55:43.102238 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:43.102144    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa04182b-7aeb-4269-96ea-b260e2d47b9e-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:43.756915 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:43.756878    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz" event={"ID":"fa04182b-7aeb-4269-96ea-b260e2d47b9e","Type":"ContainerDied","Data":"5bd1459707b746b0b6fd657722379ab62f875e4c15cad2b4b988fd2e03005462"}
Aug 04 15:55:43.757106 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:43.756932    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bd1459707b746b0b6fd657722379ab62f875e4c15cad2b4b988fd2e03005462"
Aug 04 15:55:43.757106 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:43.756889    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"
Aug 04 15:55:50.397203 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.397166    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"]
Aug 04 15:55:50.397734 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.397469    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms" podUID="fdc28f02-515b-40ff-83a1-a47af8b9378b" containerName="payload-processing" containerID="cri-o://554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665" gracePeriod=30
Aug 04 15:55:50.406999 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.406967    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"]
Aug 04 15:55:50.407298 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.407248    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d" podUID="2bda31ed-f388-4260-b963-be76cb08fe5b" containerName="payload-pre-processing" containerID="cri-o://80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da" gracePeriod=30
Aug 04 15:55:50.554338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.554273    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"]
Aug 04 15:55:50.557046 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.557012    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21cd04-5ffjz"]
Aug 04 15:55:50.690573 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.690545    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:55:50.693650 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.693616    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:55:50.775208 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.775176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnw5g\" (UniqueName: \"kubernetes.io/projected/2bda31ed-f388-4260-b963-be76cb08fe5b-kube-api-access-pnw5g\") pod \"2bda31ed-f388-4260-b963-be76cb08fe5b\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") "
Aug 04 15:55:50.775398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.775300    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bda31ed-f388-4260-b963-be76cb08fe5b-plugins-config-volume\") pod \"2bda31ed-f388-4260-b963-be76cb08fe5b\" (UID: \"2bda31ed-f388-4260-b963-be76cb08fe5b\") "
Aug 04 15:55:50.775398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.775328    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc28f02-515b-40ff-83a1-a47af8b9378b-plugins-config-volume\") pod \"fdc28f02-515b-40ff-83a1-a47af8b9378b\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") "
Aug 04 15:55:50.775398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.775353    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8p47\" (UniqueName: \"kubernetes.io/projected/fdc28f02-515b-40ff-83a1-a47af8b9378b-kube-api-access-k8p47\") pod \"fdc28f02-515b-40ff-83a1-a47af8b9378b\" (UID: \"fdc28f02-515b-40ff-83a1-a47af8b9378b\") "
Aug 04 15:55:50.775695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.775666    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc28f02-515b-40ff-83a1-a47af8b9378b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fdc28f02-515b-40ff-83a1-a47af8b9378b" (UID: "fdc28f02-515b-40ff-83a1-a47af8b9378b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:55:50.775762 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.775735    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bda31ed-f388-4260-b963-be76cb08fe5b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bda31ed-f388-4260-b963-be76cb08fe5b" (UID: "2bda31ed-f388-4260-b963-be76cb08fe5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:55:50.777378 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.777348    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bda31ed-f388-4260-b963-be76cb08fe5b-kube-api-access-pnw5g" (OuterVolumeSpecName: "kube-api-access-pnw5g") pod "2bda31ed-f388-4260-b963-be76cb08fe5b" (UID: "2bda31ed-f388-4260-b963-be76cb08fe5b"). InnerVolumeSpecName "kube-api-access-pnw5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:55:50.777499 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.777413    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdc28f02-515b-40ff-83a1-a47af8b9378b-kube-api-access-k8p47" (OuterVolumeSpecName: "kube-api-access-k8p47") pod "fdc28f02-515b-40ff-83a1-a47af8b9378b" (UID: "fdc28f02-515b-40ff-83a1-a47af8b9378b"). InnerVolumeSpecName "kube-api-access-k8p47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:55:50.789821 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.789791    2569 generic.go:356] "Generic (PLEG): container finished" podID="2bda31ed-f388-4260-b963-be76cb08fe5b" containerID="80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da" exitCode=0
Aug 04 15:55:50.789983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.789836    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"
Aug 04 15:55:50.789983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.789874    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d" event={"ID":"2bda31ed-f388-4260-b963-be76cb08fe5b","Type":"ContainerDied","Data":"80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da"}
Aug 04 15:55:50.789983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.789933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d" event={"ID":"2bda31ed-f388-4260-b963-be76cb08fe5b","Type":"ContainerDied","Data":"dcd2c741b84978dec39606656416ac6a5902fc49a990630540218d50ba2d44bd"}
Aug 04 15:55:50.789983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.789965    2569 scope.go:117] "RemoveContainer" containerID="80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da"
Aug 04 15:55:50.791281 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.791257    2569 generic.go:356] "Generic (PLEG): container finished" podID="fdc28f02-515b-40ff-83a1-a47af8b9378b" containerID="554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665" exitCode=0
Aug 04 15:55:50.791398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.791316    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"
Aug 04 15:55:50.791398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.791330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms" event={"ID":"fdc28f02-515b-40ff-83a1-a47af8b9378b","Type":"ContainerDied","Data":"554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665"}
Aug 04 15:55:50.791398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.791376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms" event={"ID":"fdc28f02-515b-40ff-83a1-a47af8b9378b","Type":"ContainerDied","Data":"965a92e8ee1f7be1fe60f8ddb26c4e9bb964a39973f6510b4b87db94dcfb9f22"}
Aug 04 15:55:50.799744 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.799686    2569 scope.go:117] "RemoveContainer" containerID="80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da"
Aug 04 15:55:50.800088 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:55:50.800060    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da\": container with ID starting with 80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da not found: ID does not exist" containerID="80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da"
Aug 04 15:55:50.800181 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.800095    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da"} err="failed to get container status \"80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da\": rpc error: code = NotFound desc = could not find container \"80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da\": container with ID starting with 80d4f372d2af55c90f14acdeb8a11b6513bb58ac6d57322b4987299cdea0e9da not found: ID does not exist"
Aug 04 15:55:50.800181 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.800114    2569 scope.go:117] "RemoveContainer" containerID="554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665"
Aug 04 15:55:50.808590 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.808566    2569 scope.go:117] "RemoveContainer" containerID="554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665"
Aug 04 15:55:50.808960 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:55:50.808926    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665\": container with ID starting with 554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665 not found: ID does not exist" containerID="554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665"
Aug 04 15:55:50.809034 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.808958    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665"} err="failed to get container status \"554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665\": rpc error: code = NotFound desc = could not find container \"554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665\": container with ID starting with 554155de485608417934abaeeb0ddbe5b9d4d74242f9a17ba147382494278665 not found: ID does not exist"
Aug 04 15:55:50.816659 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.816614    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"]
Aug 04 15:55:50.820088 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.820066    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-21cd04-69957846d6-7zrms"]
Aug 04 15:55:50.834425 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.834397    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"]
Aug 04 15:55:50.842611 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.842580    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-21cd04-759bbcf868-n675d"]
Aug 04 15:55:50.876103 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.876070    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bda31ed-f388-4260-b963-be76cb08fe5b-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:50.876103 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.876103    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fdc28f02-515b-40ff-83a1-a47af8b9378b-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:50.876308 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.876114    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8p47\" (UniqueName: \"kubernetes.io/projected/fdc28f02-515b-40ff-83a1-a47af8b9378b-kube-api-access-k8p47\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:50.876308 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:50.876123    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnw5g\" (UniqueName: \"kubernetes.io/projected/2bda31ed-f388-4260-b963-be76cb08fe5b-kube-api-access-pnw5g\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:51.965299 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:51.965266    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bda31ed-f388-4260-b963-be76cb08fe5b" path="/var/lib/kubelet/pods/2bda31ed-f388-4260-b963-be76cb08fe5b/volumes"
Aug 04 15:55:51.965743 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:51.965612    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa04182b-7aeb-4269-96ea-b260e2d47b9e" path="/var/lib/kubelet/pods/fa04182b-7aeb-4269-96ea-b260e2d47b9e/volumes"
Aug 04 15:55:51.965971 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:51.965956    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdc28f02-515b-40ff-83a1-a47af8b9378b" path="/var/lib/kubelet/pods/fdc28f02-515b-40ff-83a1-a47af8b9378b/volumes"
Aug 04 15:55:52.622012 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:52.621970    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:55:53.814892 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:53.814826    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb" podUID="8a231a83-d39f-4e80-8be5-17492f676f10" containerName="main" containerID="cri-o://6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa" gracePeriod=45
Aug 04 15:55:54.071589 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.071520    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:55:54.207225 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207188    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-tmp-dir\") pod \"8a231a83-d39f-4e80-8be5-17492f676f10\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") "
Aug 04 15:55:54.207225 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207232    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-dshm\") pod \"8a231a83-d39f-4e80-8be5-17492f676f10\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") "
Aug 04 15:55:54.207492 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207254    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2j4q\" (UniqueName: \"kubernetes.io/projected/8a231a83-d39f-4e80-8be5-17492f676f10-kube-api-access-s2j4q\") pod \"8a231a83-d39f-4e80-8be5-17492f676f10\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") "
Aug 04 15:55:54.207492 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207272    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a231a83-d39f-4e80-8be5-17492f676f10-tls-certs\") pod \"8a231a83-d39f-4e80-8be5-17492f676f10\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") "
Aug 04 15:55:54.207492 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207310    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-home\") pod \"8a231a83-d39f-4e80-8be5-17492f676f10\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") "
Aug 04 15:55:54.207492 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207327    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-model-cache\") pod \"8a231a83-d39f-4e80-8be5-17492f676f10\" (UID: \"8a231a83-d39f-4e80-8be5-17492f676f10\") "
Aug 04 15:55:54.207752 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207507    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a231a83-d39f-4e80-8be5-17492f676f10" (UID: "8a231a83-d39f-4e80-8be5-17492f676f10"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:54.207752 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207595    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-home" (OuterVolumeSpecName: "home") pod "8a231a83-d39f-4e80-8be5-17492f676f10" (UID: "8a231a83-d39f-4e80-8be5-17492f676f10"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:54.207752 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.207711    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a231a83-d39f-4e80-8be5-17492f676f10" (UID: "8a231a83-d39f-4e80-8be5-17492f676f10"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:54.209631 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.209609    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-dshm" (OuterVolumeSpecName: "dshm") pod "8a231a83-d39f-4e80-8be5-17492f676f10" (UID: "8a231a83-d39f-4e80-8be5-17492f676f10"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:54.209727 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.209701    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a231a83-d39f-4e80-8be5-17492f676f10-kube-api-access-s2j4q" (OuterVolumeSpecName: "kube-api-access-s2j4q") pod "8a231a83-d39f-4e80-8be5-17492f676f10" (UID: "8a231a83-d39f-4e80-8be5-17492f676f10"). InnerVolumeSpecName "kube-api-access-s2j4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:55:54.209783 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.209748    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a231a83-d39f-4e80-8be5-17492f676f10-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a231a83-d39f-4e80-8be5-17492f676f10" (UID: "8a231a83-d39f-4e80-8be5-17492f676f10"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:55:54.308104 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.308057    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:54.308104 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.308097    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:54.308104 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.308109    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2j4q\" (UniqueName: \"kubernetes.io/projected/8a231a83-d39f-4e80-8be5-17492f676f10-kube-api-access-s2j4q\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:54.308361 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.308124    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a231a83-d39f-4e80-8be5-17492f676f10-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:54.308361 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.308136    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:54.308361 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.308146    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a231a83-d39f-4e80-8be5-17492f676f10-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:54.723935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.723869    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b" podUID="11df3234-77f8-4ef1-822b-53edd523e4e2" containerName="main" containerID="cri-o://6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f" gracePeriod=45
Aug 04 15:55:54.819247 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.819213    2569 generic.go:356] "Generic (PLEG): container finished" podID="8a231a83-d39f-4e80-8be5-17492f676f10" containerID="6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa" exitCode=0
Aug 04 15:55:54.819868 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.819252    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"
Aug 04 15:55:54.819868 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.819304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb" event={"ID":"8a231a83-d39f-4e80-8be5-17492f676f10","Type":"ContainerDied","Data":"6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa"}
Aug 04 15:55:54.819868 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.819365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb" event={"ID":"8a231a83-d39f-4e80-8be5-17492f676f10","Type":"ContainerDied","Data":"07a393411cba973c690bc68cdd99b044336ab2fa574e9883dc2a6a2658b08a21"}
Aug 04 15:55:54.819868 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.819389    2569 scope.go:117] "RemoveContainer" containerID="6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa"
Aug 04 15:55:54.836543 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.836454    2569 scope.go:117] "RemoveContainer" containerID="6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa"
Aug 04 15:55:54.836874 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:55:54.836849    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa\": container with ID starting with 6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa not found: ID does not exist" containerID="6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa"
Aug 04 15:55:54.836957 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.836887    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa"} err="failed to get container status \"6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa\": rpc error: code = NotFound desc = could not find container \"6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa\": container with ID starting with 6e05ed17cc06aba1bded7a8005af3698cce544c553dd63f9a5984c374c5718aa not found: ID does not exist"
Aug 04 15:55:54.846065 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.846039    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"]
Aug 04 15:55:54.852446 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.852419    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-21cd04/test-model-21cd04-kserve-6c5668b4fc-gccgb"]
Aug 04 15:55:54.977104 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:54.977038    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:55:55.114897 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.114862    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-dshm\") pod \"11df3234-77f8-4ef1-822b-53edd523e4e2\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") "
Aug 04 15:55:55.114897 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.114898    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-home\") pod \"11df3234-77f8-4ef1-822b-53edd523e4e2\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") "
Aug 04 15:55:55.115138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.114932    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg8jt\" (UniqueName: \"kubernetes.io/projected/11df3234-77f8-4ef1-822b-53edd523e4e2-kube-api-access-cg8jt\") pod \"11df3234-77f8-4ef1-822b-53edd523e4e2\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") "
Aug 04 15:55:55.115138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.114967    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-model-cache\") pod \"11df3234-77f8-4ef1-822b-53edd523e4e2\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") "
Aug 04 15:55:55.115138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.114989    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-tmp-dir\") pod \"11df3234-77f8-4ef1-822b-53edd523e4e2\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") "
Aug 04 15:55:55.115138 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115047    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11df3234-77f8-4ef1-822b-53edd523e4e2-tls-certs\") pod \"11df3234-77f8-4ef1-822b-53edd523e4e2\" (UID: \"11df3234-77f8-4ef1-822b-53edd523e4e2\") "
Aug 04 15:55:55.115323 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115227    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-home" (OuterVolumeSpecName: "home") pod "11df3234-77f8-4ef1-822b-53edd523e4e2" (UID: "11df3234-77f8-4ef1-822b-53edd523e4e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:55.115323 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115268    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "11df3234-77f8-4ef1-822b-53edd523e4e2" (UID: "11df3234-77f8-4ef1-822b-53edd523e4e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:55.115433 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115364    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11df3234-77f8-4ef1-822b-53edd523e4e2" (UID: "11df3234-77f8-4ef1-822b-53edd523e4e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:55.115629 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115475    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:55.115629 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115520    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:55.115629 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.115539    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:55.117490 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.117466    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-dshm" (OuterVolumeSpecName: "dshm") pod "11df3234-77f8-4ef1-822b-53edd523e4e2" (UID: "11df3234-77f8-4ef1-822b-53edd523e4e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 15:55:55.117608 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.117539    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11df3234-77f8-4ef1-822b-53edd523e4e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11df3234-77f8-4ef1-822b-53edd523e4e2" (UID: "11df3234-77f8-4ef1-822b-53edd523e4e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 15:55:55.117608 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.117536    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11df3234-77f8-4ef1-822b-53edd523e4e2-kube-api-access-cg8jt" (OuterVolumeSpecName: "kube-api-access-cg8jt") pod "11df3234-77f8-4ef1-822b-53edd523e4e2" (UID: "11df3234-77f8-4ef1-822b-53edd523e4e2"). InnerVolumeSpecName "kube-api-access-cg8jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:55:55.217003 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.216964    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11df3234-77f8-4ef1-822b-53edd523e4e2-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:55.217003 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.216996    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11df3234-77f8-4ef1-822b-53edd523e4e2-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:55.217003 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.217005    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg8jt\" (UniqueName: \"kubernetes.io/projected/11df3234-77f8-4ef1-822b-53edd523e4e2-kube-api-access-cg8jt\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:55:55.826582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.826542    2569 generic.go:356] "Generic (PLEG): container finished" podID="11df3234-77f8-4ef1-822b-53edd523e4e2" containerID="6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f" exitCode=0
Aug 04 15:55:55.826582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.826586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b" event={"ID":"11df3234-77f8-4ef1-822b-53edd523e4e2","Type":"ContainerDied","Data":"6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f"}
Aug 04 15:55:55.827193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.826595    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"
Aug 04 15:55:55.827193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.826618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b" event={"ID":"11df3234-77f8-4ef1-822b-53edd523e4e2","Type":"ContainerDied","Data":"901edf5a0506babfc449685ceae2c9dd1cfe8431f7a544202754f0527b8862e9"}
Aug 04 15:55:55.827193 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.826660    2569 scope.go:117] "RemoveContainer" containerID="6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f"
Aug 04 15:55:55.836159 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.836134    2569 scope.go:117] "RemoveContainer" containerID="6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f"
Aug 04 15:55:55.836483 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:55:55.836456    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f\": container with ID starting with 6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f not found: ID does not exist" containerID="6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f"
Aug 04 15:55:55.836539 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.836498    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f"} err="failed to get container status \"6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f\": rpc error: code = NotFound desc = could not find container \"6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f\": container with ID starting with 6837ea1f7b14591a52ed0b5f570259b2eee550cf22cb85c96868c9d61786db4f not found: ID does not exist"
Aug 04 15:55:55.849009 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.848980    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"]
Aug 04 15:55:55.853972 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.853945    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9d3b05/test-model-9d3b05-kserve-587475677f-djt9b"]
Aug 04 15:55:55.966132 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.966093    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11df3234-77f8-4ef1-822b-53edd523e4e2" path="/var/lib/kubelet/pods/11df3234-77f8-4ef1-822b-53edd523e4e2/volumes"
Aug 04 15:55:55.966506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:55.966493    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a231a83-d39f-4e80-8be5-17492f676f10" path="/var/lib/kubelet/pods/8a231a83-d39f-4e80-8be5-17492f676f10/volumes"
Aug 04 15:55:57.671716 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.671678    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"]
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672027    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdc28f02-515b-40ff-83a1-a47af8b9378b" containerName="payload-processing"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672041    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc28f02-515b-40ff-83a1-a47af8b9378b" containerName="payload-processing"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672048    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11df3234-77f8-4ef1-822b-53edd523e4e2" containerName="main"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672053    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="11df3234-77f8-4ef1-822b-53edd523e4e2" containerName="main"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672068    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa04182b-7aeb-4269-96ea-b260e2d47b9e" containerName="revoke-keys"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672073    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa04182b-7aeb-4269-96ea-b260e2d47b9e" containerName="revoke-keys"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672086    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a231a83-d39f-4e80-8be5-17492f676f10" containerName="main"
Aug 04 15:55:57.672089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672092    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a231a83-d39f-4e80-8be5-17492f676f10" containerName="main"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672101    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bda31ed-f388-4260-b963-be76cb08fe5b" containerName="payload-pre-processing"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672106    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bda31ed-f388-4260-b963-be76cb08fe5b" containerName="payload-pre-processing"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672157    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bda31ed-f388-4260-b963-be76cb08fe5b" containerName="payload-pre-processing"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672163    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a231a83-d39f-4e80-8be5-17492f676f10" containerName="main"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672169    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="11df3234-77f8-4ef1-822b-53edd523e4e2" containerName="main"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672176    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdc28f02-515b-40ff-83a1-a47af8b9378b" containerName="payload-processing"
Aug 04 15:55:57.672345 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.672183    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa04182b-7aeb-4269-96ea-b260e2d47b9e" containerName="revoke-keys"
Aug 04 15:55:57.676497 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.676476    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:55:57.679362 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.679335    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:55:57.679515 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.679360    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:55:57.685588 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.685558    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"]
Aug 04 15:55:57.841359 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.841326    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73abe1be-b76b-4c92-af10-d2b19611205f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv\" (UID: \"73abe1be-b76b-4c92-af10-d2b19611205f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:55:57.942444 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.942345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73abe1be-b76b-4c92-af10-d2b19611205f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv\" (UID: \"73abe1be-b76b-4c92-af10-d2b19611205f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:55:57.945302 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.945281    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:55:57.954052 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.954022    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73abe1be-b76b-4c92-af10-d2b19611205f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv\" (UID: \"73abe1be-b76b-4c92-af10-d2b19611205f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:55:57.990824 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.990790    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:55:57.999097 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:57.999068    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:55:58.143572 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:58.143545    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"]
Aug 04 15:55:58.146032 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:55:58.146005    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73abe1be_b76b_4c92_af10_d2b19611205f.slice/crio-a201aa55c05a813111124e63c7c8b95c60586b54c481aa7270faf23d5a4b7f50 WatchSource:0}: Error finding container a201aa55c05a813111124e63c7c8b95c60586b54c481aa7270faf23d5a4b7f50: Status 404 returned error can't find the container with id a201aa55c05a813111124e63c7c8b95c60586b54c481aa7270faf23d5a4b7f50
Aug 04 15:55:58.847758 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:58.847727    2569 generic.go:356] "Generic (PLEG): container finished" podID="73abe1be-b76b-4c92-af10-d2b19611205f" containerID="8945cf6c5090c113c32ed674e13c8b15d711c674977ba3e3f8bbffae8b96c5a1" exitCode=0
Aug 04 15:55:58.848173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:58.847820    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv" event={"ID":"73abe1be-b76b-4c92-af10-d2b19611205f","Type":"ContainerDied","Data":"8945cf6c5090c113c32ed674e13c8b15d711c674977ba3e3f8bbffae8b96c5a1"}
Aug 04 15:55:58.848173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:58.847875    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv" event={"ID":"73abe1be-b76b-4c92-af10-d2b19611205f","Type":"ContainerStarted","Data":"a201aa55c05a813111124e63c7c8b95c60586b54c481aa7270faf23d5a4b7f50"}
Aug 04 15:55:59.992421 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:55:59.992389    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:56:00.160432 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:00.160397    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73abe1be-b76b-4c92-af10-d2b19611205f-maas-api-service-ca\") pod \"73abe1be-b76b-4c92-af10-d2b19611205f\" (UID: \"73abe1be-b76b-4c92-af10-d2b19611205f\") "
Aug 04 15:56:00.160863 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:00.160843    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73abe1be-b76b-4c92-af10-d2b19611205f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73abe1be-b76b-4c92-af10-d2b19611205f" (UID: "73abe1be-b76b-4c92-af10-d2b19611205f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:56:00.261315 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:00.261280    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73abe1be-b76b-4c92-af10-d2b19611205f-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:56:00.860618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:00.858877    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv" event={"ID":"73abe1be-b76b-4c92-af10-d2b19611205f","Type":"ContainerDied","Data":"a201aa55c05a813111124e63c7c8b95c60586b54c481aa7270faf23d5a4b7f50"}
Aug 04 15:56:00.860618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:00.858946    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a201aa55c05a813111124e63c7c8b95c60586b54c481aa7270faf23d5a4b7f50"
Aug 04 15:56:00.860618 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:00.859015    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"
Aug 04 15:56:08.265023 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.264979    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"]
Aug 04 15:56:08.265506 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.265210    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" podUID="4dc8045e-563b-4b20-984a-a20905c8fc48" containerName="payload-processing" containerID="cri-o://ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f" gracePeriod=30
Aug 04 15:56:08.275035 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.274416    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"]
Aug 04 15:56:08.275035 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.274807    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" podUID="a267891b-04a0-42ff-a155-9f746c4deeac" containerName="payload-pre-processing" containerID="cri-o://fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756" gracePeriod=30
Aug 04 15:56:08.307285 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.307213    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" podUID="a267891b-04a0-42ff-a155-9f746c4deeac" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.44:9004: connect: connection refused"
Aug 04 15:56:08.307559 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.307216    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" podUID="4dc8045e-563b-4b20-984a-a20905c8fc48" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.45:9004: connect: connection refused"
Aug 04 15:56:08.390098 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.390063    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"]
Aug 04 15:56:08.395779 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.395747    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9d3b05-fpdrv"]
Aug 04 15:56:08.581563 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.581530    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:56:08.584219 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.584196    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:56:08.740236 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.740202    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2d5n\" (UniqueName: \"kubernetes.io/projected/4dc8045e-563b-4b20-984a-a20905c8fc48-kube-api-access-j2d5n\") pod \"4dc8045e-563b-4b20-984a-a20905c8fc48\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") "
Aug 04 15:56:08.740399 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.740274    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc8045e-563b-4b20-984a-a20905c8fc48-plugins-config-volume\") pod \"4dc8045e-563b-4b20-984a-a20905c8fc48\" (UID: \"4dc8045e-563b-4b20-984a-a20905c8fc48\") "
Aug 04 15:56:08.740399 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.740373    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tjnp\" (UniqueName: \"kubernetes.io/projected/a267891b-04a0-42ff-a155-9f746c4deeac-kube-api-access-7tjnp\") pod \"a267891b-04a0-42ff-a155-9f746c4deeac\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") "
Aug 04 15:56:08.740500 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.740419    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a267891b-04a0-42ff-a155-9f746c4deeac-plugins-config-volume\") pod \"a267891b-04a0-42ff-a155-9f746c4deeac\" (UID: \"a267891b-04a0-42ff-a155-9f746c4deeac\") "
Aug 04 15:56:08.740772 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.740733    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dc8045e-563b-4b20-984a-a20905c8fc48-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4dc8045e-563b-4b20-984a-a20905c8fc48" (UID: "4dc8045e-563b-4b20-984a-a20905c8fc48"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:56:08.740916 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.740835    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a267891b-04a0-42ff-a155-9f746c4deeac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a267891b-04a0-42ff-a155-9f746c4deeac" (UID: "a267891b-04a0-42ff-a155-9f746c4deeac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:56:08.742580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.742544    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc8045e-563b-4b20-984a-a20905c8fc48-kube-api-access-j2d5n" (OuterVolumeSpecName: "kube-api-access-j2d5n") pod "4dc8045e-563b-4b20-984a-a20905c8fc48" (UID: "4dc8045e-563b-4b20-984a-a20905c8fc48"). InnerVolumeSpecName "kube-api-access-j2d5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:56:08.742697 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.742551    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a267891b-04a0-42ff-a155-9f746c4deeac-kube-api-access-7tjnp" (OuterVolumeSpecName: "kube-api-access-7tjnp") pod "a267891b-04a0-42ff-a155-9f746c4deeac" (UID: "a267891b-04a0-42ff-a155-9f746c4deeac"). InnerVolumeSpecName "kube-api-access-7tjnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:56:08.842173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.842074    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tjnp\" (UniqueName: \"kubernetes.io/projected/a267891b-04a0-42ff-a155-9f746c4deeac-kube-api-access-7tjnp\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:56:08.842173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.842107    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a267891b-04a0-42ff-a155-9f746c4deeac-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:56:08.842173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.842118    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2d5n\" (UniqueName: \"kubernetes.io/projected/4dc8045e-563b-4b20-984a-a20905c8fc48-kube-api-access-j2d5n\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:56:08.842173 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.842126    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc8045e-563b-4b20-984a-a20905c8fc48-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:56:08.898385 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.898346    2569 generic.go:356] "Generic (PLEG): container finished" podID="a267891b-04a0-42ff-a155-9f746c4deeac" containerID="fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756" exitCode=0
Aug 04 15:56:08.898582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.898392    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"
Aug 04 15:56:08.898582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.898425    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" event={"ID":"a267891b-04a0-42ff-a155-9f746c4deeac","Type":"ContainerDied","Data":"fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756"}
Aug 04 15:56:08.898582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.898476    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr" event={"ID":"a267891b-04a0-42ff-a155-9f746c4deeac","Type":"ContainerDied","Data":"85980d10be0f6e34ee280bf8ea55003679e5682610012caab89a1a4230bdd66d"}
Aug 04 15:56:08.898582 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.898497    2569 scope.go:117] "RemoveContainer" containerID="fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756"
Aug 04 15:56:08.899885 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.899861    2569 generic.go:356] "Generic (PLEG): container finished" podID="4dc8045e-563b-4b20-984a-a20905c8fc48" containerID="ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f" exitCode=0
Aug 04 15:56:08.899998 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.899904    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"
Aug 04 15:56:08.899998 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.899945    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" event={"ID":"4dc8045e-563b-4b20-984a-a20905c8fc48","Type":"ContainerDied","Data":"ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f"}
Aug 04 15:56:08.900081 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.900004    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm" event={"ID":"4dc8045e-563b-4b20-984a-a20905c8fc48","Type":"ContainerDied","Data":"6147257fe6f168ac7b1976bb9700b1956950811c7be69e58ebe3005584aa6224"}
Aug 04 15:56:08.907594 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.907479    2569 scope.go:117] "RemoveContainer" containerID="fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756"
Aug 04 15:56:08.907909 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:56:08.907880    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756\": container with ID starting with fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756 not found: ID does not exist" containerID="fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756"
Aug 04 15:56:08.908032 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.907918    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756"} err="failed to get container status \"fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756\": rpc error: code = NotFound desc = could not find container \"fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756\": container with ID starting with fd174d264a45b643c353897f81fd951a9fdd2751577ac636e0748ef145e7f756 not found: ID does not exist"
Aug 04 15:56:08.908032 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.907943    2569 scope.go:117] "RemoveContainer" containerID="ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f"
Aug 04 15:56:08.916472 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.916455    2569 scope.go:117] "RemoveContainer" containerID="ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f"
Aug 04 15:56:08.916766 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:56:08.916743    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f\": container with ID starting with ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f not found: ID does not exist" containerID="ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f"
Aug 04 15:56:08.916815 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.916774    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f"} err="failed to get container status \"ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f\": rpc error: code = NotFound desc = could not find container \"ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f\": container with ID starting with ae6cff34b9931d408a0bbe044df5824521bc9fe7d2698103e2494ca16373b03f not found: ID does not exist"
Aug 04 15:56:08.927632 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.927594    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"]
Aug 04 15:56:08.930803 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.930776    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9d3b05-f59c7d648-l5cmr"]
Aug 04 15:56:08.941250 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.941221    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"]
Aug 04 15:56:08.944515 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:08.944487    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9d3b05-7786bbb8b-7gfmm"]
Aug 04 15:56:09.965722 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:09.965690    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dc8045e-563b-4b20-984a-a20905c8fc48" path="/var/lib/kubelet/pods/4dc8045e-563b-4b20-984a-a20905c8fc48/volumes"
Aug 04 15:56:09.966169 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:09.966033    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73abe1be-b76b-4c92-af10-d2b19611205f" path="/var/lib/kubelet/pods/73abe1be-b76b-4c92-af10-d2b19611205f/volumes"
Aug 04 15:56:09.966352 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:09.966339    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a267891b-04a0-42ff-a155-9f746c4deeac" path="/var/lib/kubelet/pods/a267891b-04a0-42ff-a155-9f746c4deeac/volumes"
Aug 04 15:56:10.426510 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:10.426476    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:56:20.794402 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794369    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"]
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794771    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc8045e-563b-4b20-984a-a20905c8fc48" containerName="payload-processing"
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794786    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc8045e-563b-4b20-984a-a20905c8fc48" containerName="payload-processing"
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794795    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a267891b-04a0-42ff-a155-9f746c4deeac" containerName="payload-pre-processing"
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794800    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a267891b-04a0-42ff-a155-9f746c4deeac" containerName="payload-pre-processing"
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794809    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73abe1be-b76b-4c92-af10-d2b19611205f" containerName="revoke-keys"
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794815    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="73abe1be-b76b-4c92-af10-d2b19611205f" containerName="revoke-keys"
Aug 04 15:56:20.794887 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794885    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a267891b-04a0-42ff-a155-9f746c4deeac" containerName="payload-pre-processing"
Aug 04 15:56:20.795116 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794895    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc8045e-563b-4b20-984a-a20905c8fc48" containerName="payload-processing"
Aug 04 15:56:20.795116 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.794902    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="73abe1be-b76b-4c92-af10-d2b19611205f" containerName="revoke-keys"
Aug 04 15:56:20.799651 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.799605    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.802836 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.802800    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ddf7a7-dockercfg-h8rrv\""
Aug 04 15:56:20.802984 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.802938    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ddf7a7\""
Aug 04 15:56:20.812630 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.812602    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"]
Aug 04 15:56:20.816040 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.816015    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"]
Aug 04 15:56:20.816160 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.816119    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:20.828563 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.826927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"]
Aug 04 15:56:20.843072 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.843042    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.843235 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.843077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhf54\" (UniqueName: \"kubernetes.io/projected/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-kube-api-access-dhf54\") pod \"payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.843235 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.843116    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81486b73-f7ec-45dd-b02d-dc3dff880c02-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:20.843235 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.843140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr8jb\" (UniqueName: \"kubernetes.io/projected/81486b73-f7ec-45dd-b02d-dc3dff880c02-kube-api-access-dr8jb\") pod \"payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:20.943865 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.943822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.943865 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.943868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhf54\" (UniqueName: \"kubernetes.io/projected/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-kube-api-access-dhf54\") pod \"payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.944161 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.943911    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81486b73-f7ec-45dd-b02d-dc3dff880c02-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:20.944161 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.943939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr8jb\" (UniqueName: \"kubernetes.io/projected/81486b73-f7ec-45dd-b02d-dc3dff880c02-kube-api-access-dr8jb\") pod \"payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:20.944809 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.944782    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.944939 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.944788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81486b73-f7ec-45dd-b02d-dc3dff880c02-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:20.951799 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.951777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhf54\" (UniqueName: \"kubernetes.io/projected/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-kube-api-access-dhf54\") pod \"payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:20.951888 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:20.951820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr8jb\" (UniqueName: \"kubernetes.io/projected/81486b73-f7ec-45dd-b02d-dc3dff880c02-kube-api-access-dr8jb\") pod \"payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") " pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:21.111589 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.111481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:21.128593 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.128559    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:21.262748 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.262710    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"]
Aug 04 15:56:21.266309 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:56:21.266275    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3f1dc6_d70c_41ab_8c5c_55c71c508142.slice/crio-cfc27a288f30a78a04041bf608c61cccf07cdfb4d05f5af7ac7efe384ca20144 WatchSource:0}: Error finding container cfc27a288f30a78a04041bf608c61cccf07cdfb4d05f5af7ac7efe384ca20144: Status 404 returned error can't find the container with id cfc27a288f30a78a04041bf608c61cccf07cdfb4d05f5af7ac7efe384ca20144
Aug 04 15:56:21.292520 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.292495    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"]
Aug 04 15:56:21.299891 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:56:21.299856    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81486b73_f7ec_45dd_b02d_dc3dff880c02.slice/crio-08617220a3a794b434953ed30df523e936dc235eafff8355c98454abb4ad2431 WatchSource:0}: Error finding container 08617220a3a794b434953ed30df523e936dc235eafff8355c98454abb4ad2431: Status 404 returned error can't find the container with id 08617220a3a794b434953ed30df523e936dc235eafff8355c98454abb4ad2431
Aug 04 15:56:21.965596 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.965570    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:21.965977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.965616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7" event={"ID":"81486b73-f7ec-45dd-b02d-dc3dff880c02","Type":"ContainerStarted","Data":"db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63"}
Aug 04 15:56:21.965977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.965658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7" event={"ID":"81486b73-f7ec-45dd-b02d-dc3dff880c02","Type":"ContainerStarted","Data":"08617220a3a794b434953ed30df523e936dc235eafff8355c98454abb4ad2431"}
Aug 04 15:56:21.965977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.965683    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp" event={"ID":"2b3f1dc6-d70c-41ab-8c5c-55c71c508142","Type":"ContainerStarted","Data":"4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8"}
Aug 04 15:56:21.965977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.965702    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:21.965977 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.965715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp" event={"ID":"2b3f1dc6-d70c-41ab-8c5c-55c71c508142","Type":"ContainerStarted","Data":"cfc27a288f30a78a04041bf608c61cccf07cdfb4d05f5af7ac7efe384ca20144"}
Aug 04 15:56:21.981616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.981562    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7" podStartSLOduration=1.9815454940000001 podStartE2EDuration="1.981545494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:21.980879756 +0000 UTC m=+2724.487177812" watchObservedRunningTime="2026-08-04 15:56:21.981545494 +0000 UTC m=+2724.487843611"
Aug 04 15:56:22.001504 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:21.999821    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp" podStartSLOduration=1.999797603 podStartE2EDuration="1.999797603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:21.999382299 +0000 UTC m=+2724.505680357" watchObservedRunningTime="2026-08-04 15:56:21.999797603 +0000 UTC m=+2724.506095662"
Aug 04 15:56:22.717689 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:22.717651    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:56:32.972983 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:32.972953    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:56:32.973475 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:32.973034    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:56:50.070159 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.070120    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"]
Aug 04 15:56:50.073833 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.073813    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.076979 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.076957    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-063515-dockercfg-shn78\""
Aug 04 15:56:50.077280 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.077262    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-063515\""
Aug 04 15:56:50.088420 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.088388    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"]
Aug 04 15:56:50.102805 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.102772    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"]
Aug 04 15:56:50.106121 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.106099    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.121277 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.121249    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"]
Aug 04 15:56:50.208253 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.208168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7598\" (UniqueName: \"kubernetes.io/projected/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-kube-api-access-h7598\") pod \"payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") " pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.208253 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.208227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fbh9\" (UniqueName: \"kubernetes.io/projected/c30e5102-be3a-4d35-ae9c-65805c5254a7-kube-api-access-6fbh9\") pod \"payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.208447 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.208271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c30e5102-be3a-4d35-ae9c-65805c5254a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.208447 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.208357    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") " pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.309198 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.309151    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") " pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.309393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.309219    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7598\" (UniqueName: \"kubernetes.io/projected/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-kube-api-access-h7598\") pod \"payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") " pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.309393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.309262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fbh9\" (UniqueName: \"kubernetes.io/projected/c30e5102-be3a-4d35-ae9c-65805c5254a7-kube-api-access-6fbh9\") pod \"payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.309393 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.309298    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c30e5102-be3a-4d35-ae9c-65805c5254a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.310049 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.310025    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") " pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.310139 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.310052    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c30e5102-be3a-4d35-ae9c-65805c5254a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.325023 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.324985    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7598\" (UniqueName: \"kubernetes.io/projected/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-kube-api-access-h7598\") pod \"payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") " pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.327848 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.327821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fbh9\" (UniqueName: \"kubernetes.io/projected/c30e5102-be3a-4d35-ae9c-65805c5254a7-kube-api-access-6fbh9\") pod \"payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.384846 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.384806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:50.416910 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.416877    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:50.542107 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.542064    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"]
Aug 04 15:56:50.546132 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:56:50.546093    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30e5102_be3a_4d35_ae9c_65805c5254a7.slice/crio-04b0dce96245755b6bcd707a8b24b58049bc102a7ebb20ddf4260194335362f0 WatchSource:0}: Error finding container 04b0dce96245755b6bcd707a8b24b58049bc102a7ebb20ddf4260194335362f0: Status 404 returned error can't find the container with id 04b0dce96245755b6bcd707a8b24b58049bc102a7ebb20ddf4260194335362f0
Aug 04 15:56:50.574290 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:50.574255    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"]
Aug 04 15:56:50.589878 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:56:50.589850    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a8ddb48_5af0_4a80_95be_07b1dfc60ef1.slice/crio-141f0d1bd0adbb69abec0e05b569b00a865c50c37aac10e77fc3cba09093e9ea WatchSource:0}: Error finding container 141f0d1bd0adbb69abec0e05b569b00a865c50c37aac10e77fc3cba09093e9ea: Status 404 returned error can't find the container with id 141f0d1bd0adbb69abec0e05b569b00a865c50c37aac10e77fc3cba09093e9ea
Aug 04 15:56:51.104182 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.104143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd" event={"ID":"c30e5102-be3a-4d35-ae9c-65805c5254a7","Type":"ContainerStarted","Data":"78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc"}
Aug 04 15:56:51.104686 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.104213    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:56:51.104686 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.104239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd" event={"ID":"c30e5102-be3a-4d35-ae9c-65805c5254a7","Type":"ContainerStarted","Data":"04b0dce96245755b6bcd707a8b24b58049bc102a7ebb20ddf4260194335362f0"}
Aug 04 15:56:51.105835 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.105808    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg" event={"ID":"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1","Type":"ContainerStarted","Data":"3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d"}
Aug 04 15:56:51.105939 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.105857    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:56:51.105939 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.105872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg" event={"ID":"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1","Type":"ContainerStarted","Data":"141f0d1bd0adbb69abec0e05b569b00a865c50c37aac10e77fc3cba09093e9ea"}
Aug 04 15:56:51.160831 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.160772    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg" podStartSLOduration=1.160748851 podStartE2EDuration="1.160748851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:51.158941223 +0000 UTC m=+2753.665239283" watchObservedRunningTime="2026-08-04 15:56:51.160748851 +0000 UTC m=+2753.667046905"
Aug 04 15:56:51.161889 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:51.161849    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd" podStartSLOduration=1.161838603 podStartE2EDuration="1.161838603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:56:51.129820635 +0000 UTC m=+2753.636118692" watchObservedRunningTime="2026-08-04 15:56:51.161838603 +0000 UTC m=+2753.668136667"
Aug 04 15:56:52.919710 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:56:52.919673    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:57:02.112839 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:02.112803    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:57:02.113293 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:02.112887    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:57:16.008183 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.008148    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"]
Aug 04 15:57:16.011079 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.011061    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:16.013700 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.013675    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:57:16.013828 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.013778    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:57:16.020599 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.020571    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"]
Aug 04 15:57:16.038764 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.038725    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5bd3cd7-8308-4350-b22a-c52c1d24cc32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n\" (UID: \"c5bd3cd7-8308-4350-b22a-c52c1d24cc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:16.139691 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.139620    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5bd3cd7-8308-4350-b22a-c52c1d24cc32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n\" (UID: \"c5bd3cd7-8308-4350-b22a-c52c1d24cc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:16.140255 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.140233    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5bd3cd7-8308-4350-b22a-c52c1d24cc32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n\" (UID: \"c5bd3cd7-8308-4350-b22a-c52c1d24cc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:16.322699 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.322580    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:16.464541 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:16.464511    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"]
Aug 04 15:57:16.467522 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:57:16.467489    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bd3cd7_8308_4350_b22a_c52c1d24cc32.slice/crio-25ecaa93c023c3768c7a2ffb2ac324a052c6816afbf683c8d472ae3d4b229507 WatchSource:0}: Error finding container 25ecaa93c023c3768c7a2ffb2ac324a052c6816afbf683c8d472ae3d4b229507: Status 404 returned error can't find the container with id 25ecaa93c023c3768c7a2ffb2ac324a052c6816afbf683c8d472ae3d4b229507
Aug 04 15:57:17.229695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:17.229630    2569 generic.go:356] "Generic (PLEG): container finished" podID="c5bd3cd7-8308-4350-b22a-c52c1d24cc32" containerID="61d8e262d1c96bfca07ae83663288019d867456200d1717b664dea44fb48d2d6" exitCode=0
Aug 04 15:57:17.230147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:17.229740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n" event={"ID":"c5bd3cd7-8308-4350-b22a-c52c1d24cc32","Type":"ContainerDied","Data":"61d8e262d1c96bfca07ae83663288019d867456200d1717b664dea44fb48d2d6"}
Aug 04 15:57:17.230147 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:17.229828    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n" event={"ID":"c5bd3cd7-8308-4350-b22a-c52c1d24cc32","Type":"ContainerStarted","Data":"25ecaa93c023c3768c7a2ffb2ac324a052c6816afbf683c8d472ae3d4b229507"}
Aug 04 15:57:18.364069 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:18.364041    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:18.473821 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:18.473784    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5bd3cd7-8308-4350-b22a-c52c1d24cc32-maas-api-service-ca\") pod \"c5bd3cd7-8308-4350-b22a-c52c1d24cc32\" (UID: \"c5bd3cd7-8308-4350-b22a-c52c1d24cc32\") "
Aug 04 15:57:18.474144 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:18.474123    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5bd3cd7-8308-4350-b22a-c52c1d24cc32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5bd3cd7-8308-4350-b22a-c52c1d24cc32" (UID: "c5bd3cd7-8308-4350-b22a-c52c1d24cc32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:57:18.574857 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:18.574767    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5bd3cd7-8308-4350-b22a-c52c1d24cc32-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:19.241125 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:19.241093    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"
Aug 04 15:57:19.241125 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:19.241116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n" event={"ID":"c5bd3cd7-8308-4350-b22a-c52c1d24cc32","Type":"ContainerDied","Data":"25ecaa93c023c3768c7a2ffb2ac324a052c6816afbf683c8d472ae3d4b229507"}
Aug 04 15:57:19.241363 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:19.241172    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25ecaa93c023c3768c7a2ffb2ac324a052c6816afbf683c8d472ae3d4b229507"
Aug 04 15:57:31.535993 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.535942    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"]
Aug 04 15:57:31.538781 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.536260    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7" podUID="81486b73-f7ec-45dd-b02d-dc3dff880c02" containerName="payload-processing" containerID="cri-o://db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63" gracePeriod=30
Aug 04 15:57:31.541023 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.540994    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"]
Aug 04 15:57:31.541278 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.541231    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp" podUID="2b3f1dc6-d70c-41ab-8c5c-55c71c508142" containerName="payload-pre-processing" containerID="cri-o://4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8" gracePeriod=30
Aug 04 15:57:31.735329 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.735287    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"]
Aug 04 15:57:31.737871 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.737844    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ddf7a7-k5w4n"]
Aug 04 15:57:31.800773 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.800697    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:57:31.832290 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.832266    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:57:31.906453 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.906415    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr8jb\" (UniqueName: \"kubernetes.io/projected/81486b73-f7ec-45dd-b02d-dc3dff880c02-kube-api-access-dr8jb\") pod \"81486b73-f7ec-45dd-b02d-dc3dff880c02\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") "
Aug 04 15:57:31.906696 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.906465    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81486b73-f7ec-45dd-b02d-dc3dff880c02-plugins-config-volume\") pod \"81486b73-f7ec-45dd-b02d-dc3dff880c02\" (UID: \"81486b73-f7ec-45dd-b02d-dc3dff880c02\") "
Aug 04 15:57:31.906696 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.906487    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-plugins-config-volume\") pod \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") "
Aug 04 15:57:31.906696 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.906554    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhf54\" (UniqueName: \"kubernetes.io/projected/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-kube-api-access-dhf54\") pod \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\" (UID: \"2b3f1dc6-d70c-41ab-8c5c-55c71c508142\") "
Aug 04 15:57:31.906963 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.906936    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b3f1dc6-d70c-41ab-8c5c-55c71c508142" (UID: "2b3f1dc6-d70c-41ab-8c5c-55c71c508142"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:57:31.907050 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.907003    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81486b73-f7ec-45dd-b02d-dc3dff880c02-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81486b73-f7ec-45dd-b02d-dc3dff880c02" (UID: "81486b73-f7ec-45dd-b02d-dc3dff880c02"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:57:31.908723 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.908699    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81486b73-f7ec-45dd-b02d-dc3dff880c02-kube-api-access-dr8jb" (OuterVolumeSpecName: "kube-api-access-dr8jb") pod "81486b73-f7ec-45dd-b02d-dc3dff880c02" (UID: "81486b73-f7ec-45dd-b02d-dc3dff880c02"). InnerVolumeSpecName "kube-api-access-dr8jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:57:31.908807 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.908766    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-kube-api-access-dhf54" (OuterVolumeSpecName: "kube-api-access-dhf54") pod "2b3f1dc6-d70c-41ab-8c5c-55c71c508142" (UID: "2b3f1dc6-d70c-41ab-8c5c-55c71c508142"). InnerVolumeSpecName "kube-api-access-dhf54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:57:31.965059 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:31.965016    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5bd3cd7-8308-4350-b22a-c52c1d24cc32" path="/var/lib/kubelet/pods/c5bd3cd7-8308-4350-b22a-c52c1d24cc32/volumes"
Aug 04 15:57:32.007481 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.007441    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhf54\" (UniqueName: \"kubernetes.io/projected/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-kube-api-access-dhf54\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:32.007481 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.007474    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr8jb\" (UniqueName: \"kubernetes.io/projected/81486b73-f7ec-45dd-b02d-dc3dff880c02-kube-api-access-dr8jb\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:32.007481 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.007485    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81486b73-f7ec-45dd-b02d-dc3dff880c02-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:32.007783 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.007495    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b3f1dc6-d70c-41ab-8c5c-55c71c508142-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:32.310415 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.310372    2569 generic.go:356] "Generic (PLEG): container finished" podID="2b3f1dc6-d70c-41ab-8c5c-55c71c508142" containerID="4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8" exitCode=0
Aug 04 15:57:32.310612 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.310424    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"
Aug 04 15:57:32.310612 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.310461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp" event={"ID":"2b3f1dc6-d70c-41ab-8c5c-55c71c508142","Type":"ContainerDied","Data":"4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8"}
Aug 04 15:57:32.310612 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.310517    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp" event={"ID":"2b3f1dc6-d70c-41ab-8c5c-55c71c508142","Type":"ContainerDied","Data":"cfc27a288f30a78a04041bf608c61cccf07cdfb4d05f5af7ac7efe384ca20144"}
Aug 04 15:57:32.310612 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.310550    2569 scope.go:117] "RemoveContainer" containerID="4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8"
Aug 04 15:57:32.311904 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.311879    2569 generic.go:356] "Generic (PLEG): container finished" podID="81486b73-f7ec-45dd-b02d-dc3dff880c02" containerID="db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63" exitCode=0
Aug 04 15:57:32.311994 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.311926    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"
Aug 04 15:57:32.311994 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.311944    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7" event={"ID":"81486b73-f7ec-45dd-b02d-dc3dff880c02","Type":"ContainerDied","Data":"db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63"}
Aug 04 15:57:32.311994 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.311982    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7" event={"ID":"81486b73-f7ec-45dd-b02d-dc3dff880c02","Type":"ContainerDied","Data":"08617220a3a794b434953ed30df523e936dc235eafff8355c98454abb4ad2431"}
Aug 04 15:57:32.319851 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.319831    2569 scope.go:117] "RemoveContainer" containerID="4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8"
Aug 04 15:57:32.320125 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:57:32.320101    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8\": container with ID starting with 4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8 not found: ID does not exist" containerID="4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8"
Aug 04 15:57:32.320178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.320134    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8"} err="failed to get container status \"4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8\": rpc error: code = NotFound desc = could not find container \"4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8\": container with ID starting with 4b8c031a3b3467266d4c39a99a90ef5b8d4eda6b928394fcb8ccfdc88d74ead8 not found: ID does not exist"
Aug 04 15:57:32.320178 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.320151    2569 scope.go:117] "RemoveContainer" containerID="db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63"
Aug 04 15:57:32.328873 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.328766    2569 scope.go:117] "RemoveContainer" containerID="db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63"
Aug 04 15:57:32.329165 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:57:32.329130    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63\": container with ID starting with db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63 not found: ID does not exist" containerID="db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63"
Aug 04 15:57:32.329282 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.329185    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63"} err="failed to get container status \"db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63\": rpc error: code = NotFound desc = could not find container \"db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63\": container with ID starting with db12640552febc95b2513948dcce0bdc38208ab2509476cc219c471baa507e63 not found: ID does not exist"
Aug 04 15:57:32.329715 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.329696    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"]
Aug 04 15:57:32.335359 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.335329    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ddf7a7-8876596d-zhjgp"]
Aug 04 15:57:32.349886 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.349855    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"]
Aug 04 15:57:32.353759 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:32.353732    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ddf7a7-685bf8ff6b-rjgk7"]
Aug 04 15:57:33.965063 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:33.965027    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b3f1dc6-d70c-41ab-8c5c-55c71c508142" path="/var/lib/kubelet/pods/2b3f1dc6-d70c-41ab-8c5c-55c71c508142/volumes"
Aug 04 15:57:33.965462 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:33.965444    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81486b73-f7ec-45dd-b02d-dc3dff880c02" path="/var/lib/kubelet/pods/81486b73-f7ec-45dd-b02d-dc3dff880c02/volumes"
Aug 04 15:57:34.024402 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:34.024366    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:57:39.145035 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.144993    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"]
Aug 04 15:57:39.145451 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145361    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5bd3cd7-8308-4350-b22a-c52c1d24cc32" containerName="revoke-keys"
Aug 04 15:57:39.145451 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145374    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5bd3cd7-8308-4350-b22a-c52c1d24cc32" containerName="revoke-keys"
Aug 04 15:57:39.145451 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145386    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81486b73-f7ec-45dd-b02d-dc3dff880c02" containerName="payload-processing"
Aug 04 15:57:39.145451 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145393    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="81486b73-f7ec-45dd-b02d-dc3dff880c02" containerName="payload-processing"
Aug 04 15:57:39.145451 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145414    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3f1dc6-d70c-41ab-8c5c-55c71c508142" containerName="payload-pre-processing"
Aug 04 15:57:39.145451 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145421    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3f1dc6-d70c-41ab-8c5c-55c71c508142" containerName="payload-pre-processing"
Aug 04 15:57:39.145678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145473    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="81486b73-f7ec-45dd-b02d-dc3dff880c02" containerName="payload-processing"
Aug 04 15:57:39.145678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145484    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3f1dc6-d70c-41ab-8c5c-55c71c508142" containerName="payload-pre-processing"
Aug 04 15:57:39.145678 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.145496    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5bd3cd7-8308-4350-b22a-c52c1d24cc32" containerName="revoke-keys"
Aug 04 15:57:39.148462 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.148442    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:39.151472 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.151450    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 15:57:39.151749 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.151732    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 15:57:39.159814 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.159784    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"]
Aug 04 15:57:39.273891 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.273839    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-063515-9gz5n\" (UID: \"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:39.374371 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.374322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-063515-9gz5n\" (UID: \"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:39.375011 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.374992    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-063515-9gz5n\" (UID: \"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:39.459398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.459299    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:39.605580 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:39.605539    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"]
Aug 04 15:57:39.611005 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:57:39.610962    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc850e8b9_f1ea_4875_bb0c_88340d0a3c1e.slice/crio-768266ca7595d919828afea056a6129855cf7598825c4502234d4a77d14fe176 WatchSource:0}: Error finding container 768266ca7595d919828afea056a6129855cf7598825c4502234d4a77d14fe176: Status 404 returned error can't find the container with id 768266ca7595d919828afea056a6129855cf7598825c4502234d4a77d14fe176
Aug 04 15:57:40.352177 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:40.352145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n" event={"ID":"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e","Type":"ContainerStarted","Data":"6db567631207f2fb7ec1cc85228bacfc2398dbc891c105b1ee250b42986f93d2"}
Aug 04 15:57:40.352505 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:40.352200    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n" event={"ID":"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e","Type":"ContainerStarted","Data":"768266ca7595d919828afea056a6129855cf7598825c4502234d4a77d14fe176"}
Aug 04 15:57:41.357927 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:41.357886    2569 generic.go:356] "Generic (PLEG): container finished" podID="c850e8b9-f1ea-4875-bb0c-88340d0a3c1e" containerID="6db567631207f2fb7ec1cc85228bacfc2398dbc891c105b1ee250b42986f93d2" exitCode=0
Aug 04 15:57:41.358376 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:41.357976    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n" event={"ID":"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e","Type":"ContainerDied","Data":"6db567631207f2fb7ec1cc85228bacfc2398dbc891c105b1ee250b42986f93d2"}
Aug 04 15:57:41.493023 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:41.492997    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:41.596851 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:41.596806    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e-maas-api-service-ca\") pod \"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e\" (UID: \"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e\") "
Aug 04 15:57:41.597200 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:41.597178    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c850e8b9-f1ea-4875-bb0c-88340d0a3c1e" (UID: "c850e8b9-f1ea-4875-bb0c-88340d0a3c1e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:57:41.697969 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:41.697930    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:42.363574 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:42.363548    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"
Aug 04 15:57:42.363574 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:42.363559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n" event={"ID":"c850e8b9-f1ea-4875-bb0c-88340d0a3c1e","Type":"ContainerDied","Data":"768266ca7595d919828afea056a6129855cf7598825c4502234d4a77d14fe176"}
Aug 04 15:57:42.364123 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:42.363604    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="768266ca7595d919828afea056a6129855cf7598825c4502234d4a77d14fe176"
Aug 04 15:57:54.654248 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.654048    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"]
Aug 04 15:57:54.654808 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.654321    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg" podUID="5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" containerName="payload-processing" containerID="cri-o://3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d" gracePeriod=30
Aug 04 15:57:54.666195 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.666160    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"]
Aug 04 15:57:54.666449 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.666424    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd" podUID="c30e5102-be3a-4d35-ae9c-65805c5254a7" containerName="payload-pre-processing" containerID="cri-o://78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc" gracePeriod=30
Aug 04 15:57:54.812778 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.812710    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"]
Aug 04 15:57:54.813912 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.813887    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-063515-9gz5n"]
Aug 04 15:57:54.942242 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.942218    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:57:54.948392 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:54.948365    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:57:55.016734 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.016702    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c30e5102-be3a-4d35-ae9c-65805c5254a7-plugins-config-volume\") pod \"c30e5102-be3a-4d35-ae9c-65805c5254a7\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") "
Aug 04 15:57:55.016935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.016748    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-plugins-config-volume\") pod \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") "
Aug 04 15:57:55.016935 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.016789    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fbh9\" (UniqueName: \"kubernetes.io/projected/c30e5102-be3a-4d35-ae9c-65805c5254a7-kube-api-access-6fbh9\") pod \"c30e5102-be3a-4d35-ae9c-65805c5254a7\" (UID: \"c30e5102-be3a-4d35-ae9c-65805c5254a7\") "
Aug 04 15:57:55.017058 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.016960    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7598\" (UniqueName: \"kubernetes.io/projected/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-kube-api-access-h7598\") pod \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\" (UID: \"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1\") "
Aug 04 15:57:55.017169 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.017139    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" (UID: "5a8ddb48-5af0-4a80-95be-07b1dfc60ef1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:57:55.017256 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.017139    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c30e5102-be3a-4d35-ae9c-65805c5254a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c30e5102-be3a-4d35-ae9c-65805c5254a7" (UID: "c30e5102-be3a-4d35-ae9c-65805c5254a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 15:57:55.017256 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.017239    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c30e5102-be3a-4d35-ae9c-65805c5254a7-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:55.017256 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.017253    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:55.019064 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.019041    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30e5102-be3a-4d35-ae9c-65805c5254a7-kube-api-access-6fbh9" (OuterVolumeSpecName: "kube-api-access-6fbh9") pod "c30e5102-be3a-4d35-ae9c-65805c5254a7" (UID: "c30e5102-be3a-4d35-ae9c-65805c5254a7"). InnerVolumeSpecName "kube-api-access-6fbh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:57:55.019064 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.019048    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-kube-api-access-h7598" (OuterVolumeSpecName: "kube-api-access-h7598") pod "5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" (UID: "5a8ddb48-5af0-4a80-95be-07b1dfc60ef1"). InnerVolumeSpecName "kube-api-access-h7598". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 15:57:55.118408 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.118370    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7598\" (UniqueName: \"kubernetes.io/projected/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1-kube-api-access-h7598\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:55.118408 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.118398    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fbh9\" (UniqueName: \"kubernetes.io/projected/c30e5102-be3a-4d35-ae9c-65805c5254a7-kube-api-access-6fbh9\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 15:57:55.424024 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.423988    2569 generic.go:356] "Generic (PLEG): container finished" podID="c30e5102-be3a-4d35-ae9c-65805c5254a7" containerID="78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc" exitCode=0
Aug 04 15:57:55.424024 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.424028    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"
Aug 04 15:57:55.424266 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.424072    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd" event={"ID":"c30e5102-be3a-4d35-ae9c-65805c5254a7","Type":"ContainerDied","Data":"78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc"}
Aug 04 15:57:55.424266 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.424142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd" event={"ID":"c30e5102-be3a-4d35-ae9c-65805c5254a7","Type":"ContainerDied","Data":"04b0dce96245755b6bcd707a8b24b58049bc102a7ebb20ddf4260194335362f0"}
Aug 04 15:57:55.424266 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.424181    2569 scope.go:117] "RemoveContainer" containerID="78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc"
Aug 04 15:57:55.425695 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.425664    2569 generic.go:356] "Generic (PLEG): container finished" podID="5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" containerID="3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d" exitCode=0
Aug 04 15:57:55.425853 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.425712    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg" event={"ID":"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1","Type":"ContainerDied","Data":"3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d"}
Aug 04 15:57:55.425853 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.425728    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"
Aug 04 15:57:55.425853 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.425757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg" event={"ID":"5a8ddb48-5af0-4a80-95be-07b1dfc60ef1","Type":"ContainerDied","Data":"141f0d1bd0adbb69abec0e05b569b00a865c50c37aac10e77fc3cba09093e9ea"}
Aug 04 15:57:55.434982 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.434962    2569 scope.go:117] "RemoveContainer" containerID="78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc"
Aug 04 15:57:55.435287 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:57:55.435268    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc\": container with ID starting with 78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc not found: ID does not exist" containerID="78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc"
Aug 04 15:57:55.435334 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.435294    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc"} err="failed to get container status \"78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc\": rpc error: code = NotFound desc = could not find container \"78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc\": container with ID starting with 78664e1f2e173a4b92fe27f17df895b9eb855ba962312750fabcc2af72017dbc not found: ID does not exist"
Aug 04 15:57:55.435334 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.435309    2569 scope.go:117] "RemoveContainer" containerID="3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d"
Aug 04 15:57:55.443398 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.443375    2569 scope.go:117] "RemoveContainer" containerID="3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d"
Aug 04 15:57:55.443711 ip-10-0-136-40 kubenswrapper[2569]: E0804 15:57:55.443691    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d\": container with ID starting with 3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d not found: ID does not exist" containerID="3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d"
Aug 04 15:57:55.443771 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.443721    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d"} err="failed to get container status \"3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d\": rpc error: code = NotFound desc = could not find container \"3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d\": container with ID starting with 3c832a64bf24383a731a84e233d1fa83f408e7c07b1be2aa7e2bdb6816a0b74d not found: ID does not exist"
Aug 04 15:57:55.452000 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.451971    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"]
Aug 04 15:57:55.455880 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.455855    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-063515-77b9955668-q4mkd"]
Aug 04 15:57:55.467999 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.467966    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"]
Aug 04 15:57:55.478088 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.478054    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-063515-7bb79f7f5d-k8cqg"]
Aug 04 15:57:55.964479 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.964444    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" path="/var/lib/kubelet/pods/5a8ddb48-5af0-4a80-95be-07b1dfc60ef1/volumes"
Aug 04 15:57:55.964897 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.964832    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30e5102-be3a-4d35-ae9c-65805c5254a7" path="/var/lib/kubelet/pods/c30e5102-be3a-4d35-ae9c-65805c5254a7/volumes"
Aug 04 15:57:55.965115 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:55.965102    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c850e8b9-f1ea-4875-bb0c-88340d0a3c1e" path="/var/lib/kubelet/pods/c850e8b9-f1ea-4875-bb0c-88340d0a3c1e/volumes"
Aug 04 15:57:56.727215 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:57:56.727179    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:58:08.555549 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555513    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"]
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555882    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c850e8b9-f1ea-4875-bb0c-88340d0a3c1e" containerName="revoke-keys"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555896    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c850e8b9-f1ea-4875-bb0c-88340d0a3c1e" containerName="revoke-keys"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555909    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30e5102-be3a-4d35-ae9c-65805c5254a7" containerName="payload-pre-processing"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555915    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30e5102-be3a-4d35-ae9c-65805c5254a7" containerName="payload-pre-processing"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555924    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" containerName="payload-processing"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555929    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" containerName="payload-processing"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555989    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30e5102-be3a-4d35-ae9c-65805c5254a7" containerName="payload-pre-processing"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.555997    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c850e8b9-f1ea-4875-bb0c-88340d0a3c1e" containerName="revoke-keys"
Aug 04 15:58:08.556028 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.556004    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8ddb48-5af0-4a80-95be-07b1dfc60ef1" containerName="payload-processing"
Aug 04 15:58:08.558864 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.558847    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.561385 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.561353    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-67a15f\""
Aug 04 15:58:08.561539 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.561463    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-67a15f-dockercfg-zdnd6\""
Aug 04 15:58:08.571357 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.571326    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"]
Aug 04 15:58:08.592396 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.592364    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"]
Aug 04 15:58:08.595611 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.595593    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.605884 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.605859    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"]
Aug 04 15:58:08.637921 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.637880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcnxm\" (UniqueName: \"kubernetes.io/projected/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-kube-api-access-xcnxm\") pod \"payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.638097 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.637940    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crm22\" (UniqueName: \"kubernetes.io/projected/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-kube-api-access-crm22\") pod \"payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.638097 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.638004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.638097 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.638059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.739279 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.739240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcnxm\" (UniqueName: \"kubernetes.io/projected/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-kube-api-access-xcnxm\") pod \"payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.739483 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.739287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crm22\" (UniqueName: \"kubernetes.io/projected/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-kube-api-access-crm22\") pod \"payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.739483 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.739324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.739483 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.739343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.740356 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.740331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.740423 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.740331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.748093 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.748063    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crm22\" (UniqueName: \"kubernetes.io/projected/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-kube-api-access-crm22\") pod \"payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.748301 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.748276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcnxm\" (UniqueName: \"kubernetes.io/projected/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-kube-api-access-xcnxm\") pod \"payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:08.869194 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.869160    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:08.906625 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:08.906577    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:09.017195 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.017158    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"]
Aug 04 15:58:09.022308 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:58:09.022269    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbbbc0ac_c1b7_4d70_a5ee_d675d810bd72.slice/crio-18a2a3ee9f5ead8bd3dc024431d396b108ab42df11e5a49b77cf4864671652b1 WatchSource:0}: Error finding container 18a2a3ee9f5ead8bd3dc024431d396b108ab42df11e5a49b77cf4864671652b1: Status 404 returned error can't find the container with id 18a2a3ee9f5ead8bd3dc024431d396b108ab42df11e5a49b77cf4864671652b1
Aug 04 15:58:09.058347 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.058307    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"]
Aug 04 15:58:09.060753 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:58:09.060730    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podead24e1a_66f3_460c_8f3c_cb25c6afe89a.slice/crio-916ebc4c8d3a0f14842423743e0974f3ff848e2bae9e487974a37b5099a92e30 WatchSource:0}: Error finding container 916ebc4c8d3a0f14842423743e0974f3ff848e2bae9e487974a37b5099a92e30: Status 404 returned error can't find the container with id 916ebc4c8d3a0f14842423743e0974f3ff848e2bae9e487974a37b5099a92e30
Aug 04 15:58:09.492456 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.492352    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw" event={"ID":"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72","Type":"ContainerStarted","Data":"31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232"}
Aug 04 15:58:09.492456 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.492416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw" event={"ID":"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72","Type":"ContainerStarted","Data":"18a2a3ee9f5ead8bd3dc024431d396b108ab42df11e5a49b77cf4864671652b1"}
Aug 04 15:58:09.492456 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.492447    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:09.493987 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.493965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b" event={"ID":"ead24e1a-66f3-460c-8f3c-cb25c6afe89a","Type":"ContainerStarted","Data":"d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672"}
Aug 04 15:58:09.494062 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.494008    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:09.494062 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.494027    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b" event={"ID":"ead24e1a-66f3-460c-8f3c-cb25c6afe89a","Type":"ContainerStarted","Data":"916ebc4c8d3a0f14842423743e0974f3ff848e2bae9e487974a37b5099a92e30"}
Aug 04 15:58:09.513184 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.513126    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw" podStartSLOduration=1.5131096419999999 podStartE2EDuration="1.513109642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:58:09.511688622 +0000 UTC m=+2832.017986679" watchObservedRunningTime="2026-08-04 15:58:09.513109642 +0000 UTC m=+2832.019407698"
Aug 04 15:58:09.531502 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:09.531437    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b" podStartSLOduration=1.531411812 podStartE2EDuration="1.531411812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:58:09.528676445 +0000 UTC m=+2832.034974501" watchObservedRunningTime="2026-08-04 15:58:09.531411812 +0000 UTC m=+2832.037709869"
Aug 04 15:58:10.515699 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:10.515662    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:58:20.500299 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:20.500265    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 15:58:20.500800 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:20.500466    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 15:58:36.582477 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.582433    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"]
Aug 04 15:58:36.585968 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.585948    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.588889 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.588853    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-4d9499-dockercfg-k2wmt\""
Aug 04 15:58:36.589163 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.589149    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-4d9499\""
Aug 04 15:58:36.591229 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.591203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmjs2\" (UniqueName: \"kubernetes.io/projected/23224110-c292-4a4f-98d0-8774aa22c384-kube-api-access-cmjs2\") pod \"payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.591387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.591317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23224110-c292-4a4f-98d0-8774aa22c384-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.602761 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.602728    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"]
Aug 04 15:58:36.628551 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.628505    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"]
Aug 04 15:58:36.632573 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.632551    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.654201 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.654170    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"]
Aug 04 15:58:36.691781 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.691743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23224110-c292-4a4f-98d0-8774aa22c384-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.691781 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.691800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmjs2\" (UniqueName: \"kubernetes.io/projected/23224110-c292-4a4f-98d0-8774aa22c384-kube-api-access-cmjs2\") pod \"payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.692085 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.691846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8vkm\" (UniqueName: \"kubernetes.io/projected/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-kube-api-access-m8vkm\") pod \"payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.692085 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.691878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.692616 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.692586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23224110-c292-4a4f-98d0-8774aa22c384-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.712295 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.712256    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmjs2\" (UniqueName: \"kubernetes.io/projected/23224110-c292-4a4f-98d0-8774aa22c384-kube-api-access-cmjs2\") pod \"payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.792834 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.792798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8vkm\" (UniqueName: \"kubernetes.io/projected/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-kube-api-access-m8vkm\") pod \"payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.793041 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.792847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.793707 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.793685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.801890 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.801858    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8vkm\" (UniqueName: \"kubernetes.io/projected/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-kube-api-access-m8vkm\") pod \"payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") " pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:36.897505 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.897469    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:36.942528 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:36.942490    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:37.049861 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.049826    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"]
Aug 04 15:58:37.053452 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:58:37.053422    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23224110_c292_4a4f_98d0_8774aa22c384.slice/crio-353af99a2c6aaa4f6364a67ea2fd6e9a7a1a8d7d0610ebfa38f53ca7635ee6be WatchSource:0}: Error finding container 353af99a2c6aaa4f6364a67ea2fd6e9a7a1a8d7d0610ebfa38f53ca7635ee6be: Status 404 returned error can't find the container with id 353af99a2c6aaa4f6364a67ea2fd6e9a7a1a8d7d0610ebfa38f53ca7635ee6be
Aug 04 15:58:37.101206 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.101168    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"]
Aug 04 15:58:37.103253 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:58:37.103226    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1c0a4f_3d0a_49f0_bd9d_96e5b11105bc.slice/crio-9838ceba07a0b028eef560fb26ceead73228a54cdaeee28adcae4983ec575d21 WatchSource:0}: Error finding container 9838ceba07a0b028eef560fb26ceead73228a54cdaeee28adcae4983ec575d21: Status 404 returned error can't find the container with id 9838ceba07a0b028eef560fb26ceead73228a54cdaeee28adcae4983ec575d21
Aug 04 15:58:37.625199 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.625062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67" event={"ID":"23224110-c292-4a4f-98d0-8774aa22c384","Type":"ContainerStarted","Data":"b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57"}
Aug 04 15:58:37.625199 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.625146    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:37.625199 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.625168    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67" event={"ID":"23224110-c292-4a4f-98d0-8774aa22c384","Type":"ContainerStarted","Data":"353af99a2c6aaa4f6364a67ea2fd6e9a7a1a8d7d0610ebfa38f53ca7635ee6be"}
Aug 04 15:58:37.626921 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.626892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9" event={"ID":"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc","Type":"ContainerStarted","Data":"991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b"}
Aug 04 15:58:37.627073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.626945    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9" event={"ID":"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc","Type":"ContainerStarted","Data":"9838ceba07a0b028eef560fb26ceead73228a54cdaeee28adcae4983ec575d21"}
Aug 04 15:58:37.627073 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.626985    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:58:37.644844 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.644784    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67" podStartSLOduration=1.6447662809999999 podStartE2EDuration="1.644766281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:58:37.643791565 +0000 UTC m=+2860.150089625" watchObservedRunningTime="2026-08-04 15:58:37.644766281 +0000 UTC m=+2860.151064338"
Aug 04 15:58:37.663308 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:37.663255    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9" podStartSLOduration=1.663235335 podStartE2EDuration="1.663235335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:58:37.661108729 +0000 UTC m=+2860.167406787" watchObservedRunningTime="2026-08-04 15:58:37.663235335 +0000 UTC m=+2860.169533391"
Aug 04 15:58:39.023244 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:39.023204    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:58:48.634170 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:48.634137    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 15:58:48.634703 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:58:48.634216    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 15:59:03.678534 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.678491    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"]
Aug 04 15:59:03.683560 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.683536    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.688112 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.688085    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"kube-root-ca.crt\""
Aug 04 15:59:03.688284 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.688085    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"openshift-service-ca.crt\""
Aug 04 15:59:03.688284 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.688085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"default-dockercfg-zbw8p\""
Aug 04 15:59:03.688284 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.688085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"auth-test-model-67a15f-kserve-self-signed-certs\""
Aug 04 15:59:03.703987 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.703951    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"]
Aug 04 15:59:03.843187 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.843154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb137851-468f-4236-86d2-499ca0e7177c-tls-certs\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.843387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.843200    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-home\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.843387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.843251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-tmp-dir\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.843387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.843310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-model-cache\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.843387 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.843359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-dshm\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.843531 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.843394    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8h45\" (UniqueName: \"kubernetes.io/projected/bb137851-468f-4236-86d2-499ca0e7177c-kube-api-access-w8h45\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.944595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.944488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-dshm\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.944595 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.944554    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8h45\" (UniqueName: \"kubernetes.io/projected/bb137851-468f-4236-86d2-499ca0e7177c-kube-api-access-w8h45\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.944877 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.944610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb137851-468f-4236-86d2-499ca0e7177c-tls-certs\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.944877 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.944670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-home\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.944877 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.944731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-tmp-dir\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.944877 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.944778    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-model-cache\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.945117 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.945093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-tmp-dir\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.945190 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.945154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-model-cache\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.945247 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.945186    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-home\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.946970 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.946953    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-dshm\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.947123 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.947106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb137851-468f-4236-86d2-499ca0e7177c-tls-certs\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.954144 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.954120    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8h45\" (UniqueName: \"kubernetes.io/projected/bb137851-468f-4236-86d2-499ca0e7177c-kube-api-access-w8h45\") pod \"auth-test-model-67a15f-kserve-55954c847f-5xqdn\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") " pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:03.992586 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:03.992545    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:04.141494 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:04.141379    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"]
Aug 04 15:59:04.144400 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:59:04.144370    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb137851_468f_4236_86d2_499ca0e7177c.slice/crio-431a54f4b9bc5d5a80ca06bce1d25d0f14e11f6509c0b2403cfa75769b6af224 WatchSource:0}: Error finding container 431a54f4b9bc5d5a80ca06bce1d25d0f14e11f6509c0b2403cfa75769b6af224: Status 404 returned error can't find the container with id 431a54f4b9bc5d5a80ca06bce1d25d0f14e11f6509c0b2403cfa75769b6af224
Aug 04 15:59:04.759432 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:04.759397    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn" event={"ID":"bb137851-468f-4236-86d2-499ca0e7177c","Type":"ContainerStarted","Data":"9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d"}
Aug 04 15:59:04.759880 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:04.759441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn" event={"ID":"bb137851-468f-4236-86d2-499ca0e7177c","Type":"ContainerStarted","Data":"431a54f4b9bc5d5a80ca06bce1d25d0f14e11f6509c0b2403cfa75769b6af224"}
Aug 04 15:59:04.778261 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:04.778203    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn" podStartSLOduration=1.778185518 podStartE2EDuration="1.778185518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:59:04.777130337 +0000 UTC m=+2887.283428393" watchObservedRunningTime="2026-08-04 15:59:04.778185518 +0000 UTC m=+2887.284483574"
Aug 04 15:59:06.123109 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:06.123075    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 15:59:13.993089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:13.993048    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:13.993089 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:13.993094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:14.006294 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:14.006263    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:14.819487 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:14.819457    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 15:59:52.228355 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.228306    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"]
Aug 04 15:59:52.232449 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.232420    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.234979 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.234954    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"auth-test-model-4d9499-kserve-self-signed-certs\""
Aug 04 15:59:52.235987 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.235966    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"openshift-service-ca.crt\""
Aug 04 15:59:52.236124 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.236076    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"default-dockercfg-wzpgn\""
Aug 04 15:59:52.236124 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.236106    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"kube-root-ca.crt\""
Aug 04 15:59:52.241976 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.241953    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"]
Aug 04 15:59:52.385963 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.385919    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tmp-dir\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.386155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.385981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-model-cache\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.386155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.386083    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tls-certs\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.386155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.386103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfd97\" (UniqueName: \"kubernetes.io/projected/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-kube-api-access-cfd97\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.386155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.386123    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-home\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.386155 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.386138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-dshm\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tls-certs\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfd97\" (UniqueName: \"kubernetes.io/projected/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-kube-api-access-cfd97\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-home\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487338 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487315    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-dshm\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487712 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tmp-dir\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487712 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-model-cache\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487843 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-home\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487927 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487870    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tmp-dir\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.487927 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.487899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-model-cache\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.489851 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.489826    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-dshm\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.490023 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.490004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tls-certs\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.495917 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.495891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfd97\" (UniqueName: \"kubernetes.io/projected/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-kube-api-access-cfd97\") pod \"auth-test-model-4d9499-kserve-5b59c469ff-9cmk2\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") " pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.544001 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.543955    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 15:59:52.687868 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.687837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"]
Aug 04 15:59:52.690842 ip-10-0-136-40 kubenswrapper[2569]: W0804 15:59:52.690809    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod127a6c6f_d4c6_44d5_b674_ce7244ed5c32.slice/crio-d02651fca9d7dc41d73b8e33bddc6a80f075c45bb142afe3fb5324d15709d5e9 WatchSource:0}: Error finding container d02651fca9d7dc41d73b8e33bddc6a80f075c45bb142afe3fb5324d15709d5e9: Status 404 returned error can't find the container with id d02651fca9d7dc41d73b8e33bddc6a80f075c45bb142afe3fb5324d15709d5e9
Aug 04 15:59:52.995869 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.995762    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2" event={"ID":"127a6c6f-d4c6-44d5-b674-ce7244ed5c32","Type":"ContainerStarted","Data":"c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3"}
Aug 04 15:59:52.995869 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:52.995814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2" event={"ID":"127a6c6f-d4c6-44d5-b674-ce7244ed5c32","Type":"ContainerStarted","Data":"d02651fca9d7dc41d73b8e33bddc6a80f075c45bb142afe3fb5324d15709d5e9"}
Aug 04 15:59:53.014822 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:53.014758    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2" podStartSLOduration=1.014738504 podStartE2EDuration="1.014738504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:59:53.013862748 +0000 UTC m=+2935.520160846" watchObservedRunningTime="2026-08-04 15:59:53.014738504 +0000 UTC m=+2935.521036556"
Aug 04 15:59:55.122816 ip-10-0-136-40 kubenswrapper[2569]: I0804 15:59:55.122783    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:00:00.140040 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:00.139998    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"]
Aug 04 16:00:00.143805 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:00.143779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"
Aug 04 16:00:00.147262 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:00.147236    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 16:00:00.154483 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:00.154447    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"]
Aug 04 16:00:00.154795 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:00.154774    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"
Aug 04 16:00:00.306674 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:00.306629    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"]
Aug 04 16:00:00.309477 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:00:00.309445    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448c0fd5_1af7_4d55_aa19_10dccf1a3f44.slice/crio-6813fd08d5e62f51d0cf29a4d5e0587cc0e6bf31ecc156115c0f37c2869a9ade WatchSource:0}: Error finding container 6813fd08d5e62f51d0cf29a4d5e0587cc0e6bf31ecc156115c0f37c2869a9ade: Status 404 returned error can't find the container with id 6813fd08d5e62f51d0cf29a4d5e0587cc0e6bf31ecc156115c0f37c2869a9ade
Aug 04 16:00:01.036734 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:01.036693    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n" event={"ID":"448c0fd5-1af7-4d55-aa19-10dccf1a3f44","Type":"ContainerStarted","Data":"56f8134068642af7470668aa08208df3d06659f972791409e9a5de8c08a32327"}
Aug 04 16:00:01.036926 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:01.036760    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n" event={"ID":"448c0fd5-1af7-4d55-aa19-10dccf1a3f44","Type":"ContainerStarted","Data":"6813fd08d5e62f51d0cf29a4d5e0587cc0e6bf31ecc156115c0f37c2869a9ade"}
Aug 04 16:00:01.057330 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:01.057280    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n" podStartSLOduration=1.057263791 podStartE2EDuration="1.057263791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:00:01.055420383 +0000 UTC m=+2943.561718438" watchObservedRunningTime="2026-08-04 16:00:01.057263791 +0000 UTC m=+2943.563561847"
Aug 04 16:00:02.043596 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:02.043558    2569 generic.go:356] "Generic (PLEG): container finished" podID="448c0fd5-1af7-4d55-aa19-10dccf1a3f44" containerID="56f8134068642af7470668aa08208df3d06659f972791409e9a5de8c08a32327" exitCode=0
Aug 04 16:00:02.044064 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:02.043665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n" event={"ID":"448c0fd5-1af7-4d55-aa19-10dccf1a3f44","Type":"ContainerDied","Data":"56f8134068642af7470668aa08208df3d06659f972791409e9a5de8c08a32327"}
Aug 04 16:00:02.544551 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:02.544511    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 16:00:02.544771 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:02.544571    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 16:00:02.557600 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:02.557570    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 16:00:03.060507 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:03.060477    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 16:00:03.198938 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:03.198910    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"
Aug 04 16:00:04.054508 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:04.054477    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"
Aug 04 16:00:04.054508 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:04.054502    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n" event={"ID":"448c0fd5-1af7-4d55-aa19-10dccf1a3f44","Type":"ContainerDied","Data":"6813fd08d5e62f51d0cf29a4d5e0587cc0e6bf31ecc156115c0f37c2869a9ade"}
Aug 04 16:00:04.054748 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:04.054549    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6813fd08d5e62f51d0cf29a4d5e0587cc0e6bf31ecc156115c0f37c2869a9ade"
Aug 04 16:00:21.133536 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:21.133448    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:00:36.724426 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:36.724391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:00:47.626736 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:00:47.626691    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:01:03.022105 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:01:03.022065    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:01:18.821192 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:01:18.821152    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:01:29.828716 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:01:29.828673    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:01:45.227313 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:01:45.227276    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:01:58.308299 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:01:58.308219    2569 scope.go:117] "RemoveContainer" containerID="8945cf6c5090c113c32ed674e13c8b15d711c674977ba3e3f8bbffae8b96c5a1"
Aug 04 16:01:58.316924 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:01:58.316901    2569 scope.go:117] "RemoveContainer" containerID="9054b335c437b768d10e9a406456da460b0f03b6fee5257fca3e019a84e71f1b"
Aug 04 16:02:00.931112 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:02:00.931075    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:02:11.421497 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:02:11.421459    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:02:27.336433 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:02:27.336393    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:02:43.355194 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:02:43.355160    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:02:53.928070 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:02:53.928035    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:03:09.416731 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:03:09.416689    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:03:25.521859 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:03:25.521772    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:03:35.934089 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:03:35.934053    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:03:52.337669 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:03:52.337617    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:03:58.350059 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:03:58.350020    2569 scope.go:117] "RemoveContainer" containerID="6db567631207f2fb7ec1cc85228bacfc2398dbc891c105b1ee250b42986f93d2"
Aug 04 16:03:58.358858 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:03:58.358832    2569 scope.go:117] "RemoveContainer" containerID="61d8e262d1c96bfca07ae83663288019d867456200d1717b664dea44fb48d2d6"
Aug 04 16:04:07.317492 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:07.317452    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:04:10.927447 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:10.927407    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"]
Aug 04 16:04:11.791407 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:11.791369    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"]
Aug 04 16:04:17.717616 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:17.717574    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:04:25.946967 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:25.946906    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn" podUID="bb137851-468f-4236-86d2-499ca0e7177c" containerName="main" containerID="cri-o://9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d" gracePeriod=45
Aug 04 16:04:26.203491 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.203415    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 16:04:26.325436 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.325399    2569 generic.go:356] "Generic (PLEG): container finished" podID="bb137851-468f-4236-86d2-499ca0e7177c" containerID="9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d" exitCode=0
Aug 04 16:04:26.325671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.325455    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"
Aug 04 16:04:26.325671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.325484    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn" event={"ID":"bb137851-468f-4236-86d2-499ca0e7177c","Type":"ContainerDied","Data":"9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d"}
Aug 04 16:04:26.325671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.325529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn" event={"ID":"bb137851-468f-4236-86d2-499ca0e7177c","Type":"ContainerDied","Data":"431a54f4b9bc5d5a80ca06bce1d25d0f14e11f6509c0b2403cfa75769b6af224"}
Aug 04 16:04:26.325671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.325548    2569 scope.go:117] "RemoveContainer" containerID="9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d"
Aug 04 16:04:26.334105 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.334082    2569 scope.go:117] "RemoveContainer" containerID="9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d"
Aug 04 16:04:26.334433 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:04:26.334407    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d\": container with ID starting with 9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d not found: ID does not exist" containerID="9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d"
Aug 04 16:04:26.334520 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.334438    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d"} err="failed to get container status \"9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d\": rpc error: code = NotFound desc = could not find container \"9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d\": container with ID starting with 9012bcb87437d20ab07b786c0ddcb7a85e20ec4706cf08d41b1be097c0f7b07d not found: ID does not exist"
Aug 04 16:04:26.357305 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357276    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8h45\" (UniqueName: \"kubernetes.io/projected/bb137851-468f-4236-86d2-499ca0e7177c-kube-api-access-w8h45\") pod \"bb137851-468f-4236-86d2-499ca0e7177c\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") "
Aug 04 16:04:26.357481 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357328    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-model-cache\") pod \"bb137851-468f-4236-86d2-499ca0e7177c\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") "
Aug 04 16:04:26.357481 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357349    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-tmp-dir\") pod \"bb137851-468f-4236-86d2-499ca0e7177c\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") "
Aug 04 16:04:26.357481 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357416    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-home\") pod \"bb137851-468f-4236-86d2-499ca0e7177c\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") "
Aug 04 16:04:26.357481 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357447    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-dshm\") pod \"bb137851-468f-4236-86d2-499ca0e7177c\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") "
Aug 04 16:04:26.357481 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357470    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb137851-468f-4236-86d2-499ca0e7177c-tls-certs\") pod \"bb137851-468f-4236-86d2-499ca0e7177c\" (UID: \"bb137851-468f-4236-86d2-499ca0e7177c\") "
Aug 04 16:04:26.357780 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357587    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb137851-468f-4236-86d2-499ca0e7177c" (UID: "bb137851-468f-4236-86d2-499ca0e7177c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:26.357780 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357692    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb137851-468f-4236-86d2-499ca0e7177c" (UID: "bb137851-468f-4236-86d2-499ca0e7177c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:26.357866 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357806    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-home" (OuterVolumeSpecName: "home") pod "bb137851-468f-4236-86d2-499ca0e7177c" (UID: "bb137851-468f-4236-86d2-499ca0e7177c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:26.357910 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357880    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:26.357910 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.357900    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:26.359812 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.359787    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-dshm" (OuterVolumeSpecName: "dshm") pod "bb137851-468f-4236-86d2-499ca0e7177c" (UID: "bb137851-468f-4236-86d2-499ca0e7177c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:26.359812 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.359806    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb137851-468f-4236-86d2-499ca0e7177c-kube-api-access-w8h45" (OuterVolumeSpecName: "kube-api-access-w8h45") pod "bb137851-468f-4236-86d2-499ca0e7177c" (UID: "bb137851-468f-4236-86d2-499ca0e7177c"). InnerVolumeSpecName "kube-api-access-w8h45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:04:26.359957 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.359831    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb137851-468f-4236-86d2-499ca0e7177c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb137851-468f-4236-86d2-499ca0e7177c" (UID: "bb137851-468f-4236-86d2-499ca0e7177c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 16:04:26.459429 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.459317    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8h45\" (UniqueName: \"kubernetes.io/projected/bb137851-468f-4236-86d2-499ca0e7177c-kube-api-access-w8h45\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:26.459429 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.459367    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:26.459429 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.459379    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb137851-468f-4236-86d2-499ca0e7177c-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:26.459429 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.459387    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb137851-468f-4236-86d2-499ca0e7177c-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:26.656355 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.656315    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"]
Aug 04 16:04:26.664738 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.664700    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/auth-test-model-67a15f-kserve-55954c847f-5xqdn"]
Aug 04 16:04:26.811096 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:26.810982    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2" podUID="127a6c6f-d4c6-44d5-b674-ce7244ed5c32" containerName="main" containerID="cri-o://c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3" gracePeriod=45
Aug 04 16:04:27.064463 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.064387    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 16:04:27.165990 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.165954    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-home\") pod \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") "
Aug 04 16:04:27.166184 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfd97\" (UniqueName: \"kubernetes.io/projected/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-kube-api-access-cfd97\") pod \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") "
Aug 04 16:04:27.166184 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166054    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-dshm\") pod \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") "
Aug 04 16:04:27.166184 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166070    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tls-certs\") pod \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") "
Aug 04 16:04:27.166184 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166128    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tmp-dir\") pod \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") "
Aug 04 16:04:27.166184 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166165    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-model-cache\") pod \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\" (UID: \"127a6c6f-d4c6-44d5-b674-ce7244ed5c32\") "
Aug 04 16:04:27.166460 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166245    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-home" (OuterVolumeSpecName: "home") pod "127a6c6f-d4c6-44d5-b674-ce7244ed5c32" (UID: "127a6c6f-d4c6-44d5-b674-ce7244ed5c32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:27.166523 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166474    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:27.166523 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166470    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-model-cache" (OuterVolumeSpecName: "model-cache") pod "127a6c6f-d4c6-44d5-b674-ce7244ed5c32" (UID: "127a6c6f-d4c6-44d5-b674-ce7244ed5c32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:27.166523 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.166484    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "127a6c6f-d4c6-44d5-b674-ce7244ed5c32" (UID: "127a6c6f-d4c6-44d5-b674-ce7244ed5c32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:27.168447 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.168409    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-dshm" (OuterVolumeSpecName: "dshm") pod "127a6c6f-d4c6-44d5-b674-ce7244ed5c32" (UID: "127a6c6f-d4c6-44d5-b674-ce7244ed5c32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:04:27.168570 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.168464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "127a6c6f-d4c6-44d5-b674-ce7244ed5c32" (UID: "127a6c6f-d4c6-44d5-b674-ce7244ed5c32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 16:04:27.168570 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.168476    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-kube-api-access-cfd97" (OuterVolumeSpecName: "kube-api-access-cfd97") pod "127a6c6f-d4c6-44d5-b674-ce7244ed5c32" (UID: "127a6c6f-d4c6-44d5-b674-ce7244ed5c32"). InnerVolumeSpecName "kube-api-access-cfd97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:04:27.267782 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.267742    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfd97\" (UniqueName: \"kubernetes.io/projected/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-kube-api-access-cfd97\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:27.267782 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.267776    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:27.267782 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.267785    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:27.268033 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.267797    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:27.268033 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.267810    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/127a6c6f-d4c6-44d5-b674-ce7244ed5c32-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:04:27.332167 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.332079    2569 generic.go:356] "Generic (PLEG): container finished" podID="127a6c6f-d4c6-44d5-b674-ce7244ed5c32" containerID="c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3" exitCode=0
Aug 04 16:04:27.332167 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.332115    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2" event={"ID":"127a6c6f-d4c6-44d5-b674-ce7244ed5c32","Type":"ContainerDied","Data":"c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3"}
Aug 04 16:04:27.332167 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.332123    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"
Aug 04 16:04:27.332167 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.332142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2" event={"ID":"127a6c6f-d4c6-44d5-b674-ce7244ed5c32","Type":"ContainerDied","Data":"d02651fca9d7dc41d73b8e33bddc6a80f075c45bb142afe3fb5324d15709d5e9"}
Aug 04 16:04:27.332167 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.332158    2569 scope.go:117] "RemoveContainer" containerID="c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3"
Aug 04 16:04:27.341907 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.341889    2569 scope.go:117] "RemoveContainer" containerID="c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3"
Aug 04 16:04:27.342214 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:04:27.342194    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3\": container with ID starting with c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3 not found: ID does not exist" containerID="c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3"
Aug 04 16:04:27.342260 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.342237    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3"} err="failed to get container status \"c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3\": rpc error: code = NotFound desc = could not find container \"c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3\": container with ID starting with c21feb061f05fd46d8f1ed27509266f96165925f849f3a13c8cc2d0a92cd3cf3 not found: ID does not exist"
Aug 04 16:04:27.364679 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.364519    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"]
Aug 04 16:04:27.372576 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.372540    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/auth-test-model-4d9499-kserve-5b59c469ff-9cmk2"]
Aug 04 16:04:27.965117 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.965084    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="127a6c6f-d4c6-44d5-b674-ce7244ed5c32" path="/var/lib/kubelet/pods/127a6c6f-d4c6-44d5-b674-ce7244ed5c32/volumes"
Aug 04 16:04:27.965492 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:27.965479    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb137851-468f-4236-86d2-499ca0e7177c" path="/var/lib/kubelet/pods/bb137851-468f-4236-86d2-499ca0e7177c/volumes"
Aug 04 16:04:40.810308 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.810261    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"]
Aug 04 16:04:40.812235 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.811941    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127a6c6f-d4c6-44d5-b674-ce7244ed5c32" containerName="main"
Aug 04 16:04:40.812235 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.811974    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="127a6c6f-d4c6-44d5-b674-ce7244ed5c32" containerName="main"
Aug 04 16:04:40.812235 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812004    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="448c0fd5-1af7-4d55-aa19-10dccf1a3f44" containerName="cleanup"
Aug 04 16:04:40.812235 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812012    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="448c0fd5-1af7-4d55-aa19-10dccf1a3f44" containerName="cleanup"
Aug 04 16:04:40.812235 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812041    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb137851-468f-4236-86d2-499ca0e7177c" containerName="main"
Aug 04 16:04:40.812235 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812049    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb137851-468f-4236-86d2-499ca0e7177c" containerName="main"
Aug 04 16:04:40.813678 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812684    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="448c0fd5-1af7-4d55-aa19-10dccf1a3f44" containerName="cleanup"
Aug 04 16:04:40.813678 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812708    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb137851-468f-4236-86d2-499ca0e7177c" containerName="main"
Aug 04 16:04:40.813678 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.812730    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="127a6c6f-d4c6-44d5-b674-ce7244ed5c32" containerName="main"
Aug 04 16:04:40.817367 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.817342    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.820982 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.820953    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"sub-test-model-67a15f-kserve-self-signed-certs\""
Aug 04 16:04:40.821147 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.820959    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"default-dockercfg-zbw8p\""
Aug 04 16:04:40.821545 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.821521    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"openshift-service-ca.crt\""
Aug 04 16:04:40.821856 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.821838    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67a15f\"/\"kube-root-ca.crt\""
Aug 04 16:04:40.822741 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.822717    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"]
Aug 04 16:04:40.882926 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.882888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-home\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.883127 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.882933    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-dshm\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.883127 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.882971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjz4r\" (UniqueName: \"kubernetes.io/projected/d49929c3-2cbe-40da-9a10-f43a995c3964-kube-api-access-rjz4r\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.883127 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.883027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-model-cache\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.883312 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.883144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49929c3-2cbe-40da-9a10-f43a995c3964-tls-certs\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.883312 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.883176    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-tmp-dir\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984061 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-tmp-dir\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984061 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984072    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-home\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984343 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-dshm\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984343 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjz4r\" (UniqueName: \"kubernetes.io/projected/d49929c3-2cbe-40da-9a10-f43a995c3964-kube-api-access-rjz4r\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984343 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984294    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-model-cache\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984514 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-home\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984514 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49929c3-2cbe-40da-9a10-f43a995c3964-tls-certs\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984514 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984491    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-tmp-dir\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.984696 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.984676    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-model-cache\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.986601 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.986568    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-dshm\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:40.986850 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:40.986835    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49929c3-2cbe-40da-9a10-f43a995c3964-tls-certs\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:41.005704 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:41.005672    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjz4r\" (UniqueName: \"kubernetes.io/projected/d49929c3-2cbe-40da-9a10-f43a995c3964-kube-api-access-rjz4r\") pod \"sub-test-model-67a15f-kserve-657bb8fddc-8v66r\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") " pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:41.129548 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:41.129509    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:41.271850 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:41.271819    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"]
Aug 04 16:04:41.274616 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:04:41.274589    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd49929c3_2cbe_40da_9a10_f43a995c3964.slice/crio-7f77192b4a11acb78e1c968ff4b0cff725d51c1735582b6990aa393cd2b73149 WatchSource:0}: Error finding container 7f77192b4a11acb78e1c968ff4b0cff725d51c1735582b6990aa393cd2b73149: Status 404 returned error can't find the container with id 7f77192b4a11acb78e1c968ff4b0cff725d51c1735582b6990aa393cd2b73149
Aug 04 16:04:41.399513 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:41.399416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r" event={"ID":"d49929c3-2cbe-40da-9a10-f43a995c3964","Type":"ContainerStarted","Data":"a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc"}
Aug 04 16:04:41.399513 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:41.399472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r" event={"ID":"d49929c3-2cbe-40da-9a10-f43a995c3964","Type":"ContainerStarted","Data":"7f77192b4a11acb78e1c968ff4b0cff725d51c1735582b6990aa393cd2b73149"}
Aug 04 16:04:41.421462 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:41.421409    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r" podStartSLOduration=1.421389501 podStartE2EDuration="1.421389501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:04:41.420068329 +0000 UTC m=+3223.926366386" watchObservedRunningTime="2026-08-04 16:04:41.421389501 +0000 UTC m=+3223.927687558"
Aug 04 16:04:43.237086 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:43.237052    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:04:51.130057 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:51.129956    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:51.130057 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:51.130016    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:51.142750 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:51.142719    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:04:51.457612 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:04:51.457522    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:05:02.321615 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.321574    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"]
Aug 04 16:05:02.326437 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.326407    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.330561 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.330534    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"kube-root-ca.crt\""
Aug 04 16:05:02.331442 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.331417    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"sub-test-model-4d9499-kserve-self-signed-certs\""
Aug 04 16:05:02.331591 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.331509    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"default-dockercfg-wzpgn\""
Aug 04 16:05:02.331591 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.331573    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"openshift-service-ca.crt\""
Aug 04 16:05:02.342764 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.342734    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"]
Aug 04 16:05:02.379716 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.379671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-home\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.379716 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.379717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60743b2-e049-4369-a608-ed659334d8af-tls-certs\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.379934 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.379829    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-tmp-dir\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.379934 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.379855    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnq6r\" (UniqueName: \"kubernetes.io/projected/d60743b2-e049-4369-a608-ed659334d8af-kube-api-access-bnq6r\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.380040 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.379961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-model-cache\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.380040 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.379982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-dshm\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.480870 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.480820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-model-cache\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.480870 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.480873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-dshm\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481153 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.480905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-home\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481153 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.480935    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60743b2-e049-4369-a608-ed659334d8af-tls-certs\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481153 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.481008    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-tmp-dir\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481153 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.481033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnq6r\" (UniqueName: \"kubernetes.io/projected/d60743b2-e049-4369-a608-ed659334d8af-kube-api-access-bnq6r\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.481260    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-model-cache\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.481307    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-home\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.481451 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.481397    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-tmp-dir\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.483292 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.483261    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-dshm\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.483529 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.483512    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60743b2-e049-4369-a608-ed659334d8af-tls-certs\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.492772 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.492743    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnq6r\" (UniqueName: \"kubernetes.io/projected/d60743b2-e049-4369-a608-ed659334d8af-kube-api-access-bnq6r\") pod \"sub-test-model-4d9499-kserve-5cd496c7b5-g4swm\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") " pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.636231 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.636189    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:02.782889 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:02.782850    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"]
Aug 04 16:05:02.786033 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:05:02.786003    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd60743b2_e049_4369_a608_ed659334d8af.slice/crio-72322d0489e014d563efe945e041521ba3eff3f1535e92805759124d05300b30 WatchSource:0}: Error finding container 72322d0489e014d563efe945e041521ba3eff3f1535e92805759124d05300b30: Status 404 returned error can't find the container with id 72322d0489e014d563efe945e041521ba3eff3f1535e92805759124d05300b30
Aug 04 16:05:03.504240 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:03.504201    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm" event={"ID":"d60743b2-e049-4369-a608-ed659334d8af","Type":"ContainerStarted","Data":"12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65"}
Aug 04 16:05:03.504710 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:03.504250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm" event={"ID":"d60743b2-e049-4369-a608-ed659334d8af","Type":"ContainerStarted","Data":"72322d0489e014d563efe945e041521ba3eff3f1535e92805759124d05300b30"}
Aug 04 16:05:03.530779 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:03.530724    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm" podStartSLOduration=1.530706689 podStartE2EDuration="1.530706689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:05:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:05:03.529495841 +0000 UTC m=+3246.035793889" watchObservedRunningTime="2026-08-04 16:05:03.530706689 +0000 UTC m=+3246.037004746"
Aug 04 16:05:05.012088 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:05.012049    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:05:12.636760 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:12.636703    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:12.636760 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:12.636774    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:12.649691 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:12.649664    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:13.563480 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:13.563441    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:05:50.225884 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:05:50.225847    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:06:01.637702 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:01.637665    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:06:29.538306 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:29.538216    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:06:34.548345 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:34.548309    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"]
Aug 04 16:06:35.447205 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:35.447168    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"]
Aug 04 16:06:40.233648 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:40.233591    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:06:48.123632 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:48.123591    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:06:49.567770 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.567702    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r" podUID="d49929c3-2cbe-40da-9a10-f43a995c3964" containerName="main" containerID="cri-o://a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc" gracePeriod=45
Aug 04 16:06:49.814383 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.814355    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:06:49.999550 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999514    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-model-cache\") pod \"d49929c3-2cbe-40da-9a10-f43a995c3964\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") "
Aug 04 16:06:49.999891 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999589    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-home\") pod \"d49929c3-2cbe-40da-9a10-f43a995c3964\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") "
Aug 04 16:06:49.999891 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999629    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjz4r\" (UniqueName: \"kubernetes.io/projected/d49929c3-2cbe-40da-9a10-f43a995c3964-kube-api-access-rjz4r\") pod \"d49929c3-2cbe-40da-9a10-f43a995c3964\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") "
Aug 04 16:06:49.999891 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999683    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-dshm\") pod \"d49929c3-2cbe-40da-9a10-f43a995c3964\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") "
Aug 04 16:06:49.999891 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999722    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49929c3-2cbe-40da-9a10-f43a995c3964-tls-certs\") pod \"d49929c3-2cbe-40da-9a10-f43a995c3964\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") "
Aug 04 16:06:49.999891 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999790    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-tmp-dir\") pod \"d49929c3-2cbe-40da-9a10-f43a995c3964\" (UID: \"d49929c3-2cbe-40da-9a10-f43a995c3964\") "
Aug 04 16:06:50.000188 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999910    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-home" (OuterVolumeSpecName: "home") pod "d49929c3-2cbe-40da-9a10-f43a995c3964" (UID: "d49929c3-2cbe-40da-9a10-f43a995c3964"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.000188 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:49.999925    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-model-cache" (OuterVolumeSpecName: "model-cache") pod "d49929c3-2cbe-40da-9a10-f43a995c3964" (UID: "d49929c3-2cbe-40da-9a10-f43a995c3964"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.000188 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.000136    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d49929c3-2cbe-40da-9a10-f43a995c3964" (UID: "d49929c3-2cbe-40da-9a10-f43a995c3964"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.000326 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.000231    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.000326 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.000248    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.000326 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.000262    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.002151 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.002122    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49929c3-2cbe-40da-9a10-f43a995c3964-kube-api-access-rjz4r" (OuterVolumeSpecName: "kube-api-access-rjz4r") pod "d49929c3-2cbe-40da-9a10-f43a995c3964" (UID: "d49929c3-2cbe-40da-9a10-f43a995c3964"). InnerVolumeSpecName "kube-api-access-rjz4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:06:50.002320 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.002296    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-dshm" (OuterVolumeSpecName: "dshm") pod "d49929c3-2cbe-40da-9a10-f43a995c3964" (UID: "d49929c3-2cbe-40da-9a10-f43a995c3964"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.002320 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.002306    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d49929c3-2cbe-40da-9a10-f43a995c3964-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d49929c3-2cbe-40da-9a10-f43a995c3964" (UID: "d49929c3-2cbe-40da-9a10-f43a995c3964"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 16:06:50.004434 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.004406    2569 generic.go:356] "Generic (PLEG): container finished" podID="d49929c3-2cbe-40da-9a10-f43a995c3964" containerID="a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc" exitCode=0
Aug 04 16:06:50.004554 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.004461    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"
Aug 04 16:06:50.004554 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.004484    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r" event={"ID":"d49929c3-2cbe-40da-9a10-f43a995c3964","Type":"ContainerDied","Data":"a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc"}
Aug 04 16:06:50.004554 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.004532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r" event={"ID":"d49929c3-2cbe-40da-9a10-f43a995c3964","Type":"ContainerDied","Data":"7f77192b4a11acb78e1c968ff4b0cff725d51c1735582b6990aa393cd2b73149"}
Aug 04 16:06:50.004554 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.004551    2569 scope.go:117] "RemoveContainer" containerID="a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc"
Aug 04 16:06:50.021294 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.021274    2569 scope.go:117] "RemoveContainer" containerID="a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc"
Aug 04 16:06:50.021616 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:06:50.021593    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc\": container with ID starting with a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc not found: ID does not exist" containerID="a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc"
Aug 04 16:06:50.021697 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.021625    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc"} err="failed to get container status \"a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc\": rpc error: code = NotFound desc = could not find container \"a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc\": container with ID starting with a74f7a57a17c63d344ebc627f6f2738b72c4ddfd7e8c1f3b30d8633fa7e2d6bc not found: ID does not exist"
Aug 04 16:06:50.034192 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.034162    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"]
Aug 04 16:06:50.037370 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.037343    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-67a15f/sub-test-model-67a15f-kserve-657bb8fddc-8v66r"]
Aug 04 16:06:50.101519 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.101483    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjz4r\" (UniqueName: \"kubernetes.io/projected/d49929c3-2cbe-40da-9a10-f43a995c3964-kube-api-access-rjz4r\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.101519 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.101514    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d49929c3-2cbe-40da-9a10-f43a995c3964-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.101519 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.101526    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d49929c3-2cbe-40da-9a10-f43a995c3964-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.466850 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.466806    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm" podUID="d60743b2-e049-4369-a608-ed659334d8af" containerName="main" containerID="cri-o://12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65" gracePeriod=45
Aug 04 16:06:50.732087 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.732007    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:06:50.808957 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.808922    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnq6r\" (UniqueName: \"kubernetes.io/projected/d60743b2-e049-4369-a608-ed659334d8af-kube-api-access-bnq6r\") pod \"d60743b2-e049-4369-a608-ed659334d8af\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") "
Aug 04 16:06:50.808957 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.808960    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-dshm\") pod \"d60743b2-e049-4369-a608-ed659334d8af\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") "
Aug 04 16:06:50.809175 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.809004    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60743b2-e049-4369-a608-ed659334d8af-tls-certs\") pod \"d60743b2-e049-4369-a608-ed659334d8af\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") "
Aug 04 16:06:50.809175 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.809100    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-model-cache\") pod \"d60743b2-e049-4369-a608-ed659334d8af\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") "
Aug 04 16:06:50.809378 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.809356    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-model-cache" (OuterVolumeSpecName: "model-cache") pod "d60743b2-e049-4369-a608-ed659334d8af" (UID: "d60743b2-e049-4369-a608-ed659334d8af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.809457 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.809443    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.811258 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.811231    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d60743b2-e049-4369-a608-ed659334d8af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d60743b2-e049-4369-a608-ed659334d8af" (UID: "d60743b2-e049-4369-a608-ed659334d8af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 16:06:50.811681 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.811631    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-dshm" (OuterVolumeSpecName: "dshm") pod "d60743b2-e049-4369-a608-ed659334d8af" (UID: "d60743b2-e049-4369-a608-ed659334d8af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.811766 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.811664    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60743b2-e049-4369-a608-ed659334d8af-kube-api-access-bnq6r" (OuterVolumeSpecName: "kube-api-access-bnq6r") pod "d60743b2-e049-4369-a608-ed659334d8af" (UID: "d60743b2-e049-4369-a608-ed659334d8af"). InnerVolumeSpecName "kube-api-access-bnq6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:06:50.910064 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910034    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-tmp-dir\") pod \"d60743b2-e049-4369-a608-ed659334d8af\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") "
Aug 04 16:06:50.910064 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910069    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-home\") pod \"d60743b2-e049-4369-a608-ed659334d8af\" (UID: \"d60743b2-e049-4369-a608-ed659334d8af\") "
Aug 04 16:06:50.910337 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910219    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnq6r\" (UniqueName: \"kubernetes.io/projected/d60743b2-e049-4369-a608-ed659334d8af-kube-api-access-bnq6r\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.910337 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910229    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.910337 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910238    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d60743b2-e049-4369-a608-ed659334d8af-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:50.910461 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910336    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d60743b2-e049-4369-a608-ed659334d8af" (UID: "d60743b2-e049-4369-a608-ed659334d8af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:50.910461 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:50.910351    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-home" (OuterVolumeSpecName: "home") pod "d60743b2-e049-4369-a608-ed659334d8af" (UID: "d60743b2-e049-4369-a608-ed659334d8af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:06:51.010821 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.010725    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:51.010821 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.010749    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d60743b2-e049-4369-a608-ed659334d8af-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:06:51.012336 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.012305    2569 generic.go:356] "Generic (PLEG): container finished" podID="d60743b2-e049-4369-a608-ed659334d8af" containerID="12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65" exitCode=0
Aug 04 16:06:51.012465 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.012356    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"
Aug 04 16:06:51.012465 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.012385    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm" event={"ID":"d60743b2-e049-4369-a608-ed659334d8af","Type":"ContainerDied","Data":"12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65"}
Aug 04 16:06:51.012465 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.012429    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm" event={"ID":"d60743b2-e049-4369-a608-ed659334d8af","Type":"ContainerDied","Data":"72322d0489e014d563efe945e041521ba3eff3f1535e92805759124d05300b30"}
Aug 04 16:06:51.012465 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.012446    2569 scope.go:117] "RemoveContainer" containerID="12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65"
Aug 04 16:06:51.021951 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.021931    2569 scope.go:117] "RemoveContainer" containerID="12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65"
Aug 04 16:06:51.022224 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:06:51.022202    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65\": container with ID starting with 12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65 not found: ID does not exist" containerID="12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65"
Aug 04 16:06:51.022296 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.022238    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65"} err="failed to get container status \"12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65\": rpc error: code = NotFound desc = could not find container \"12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65\": container with ID starting with 12190f9ed23073064274ee31b90ed5375f83ce6a339e697adcf3a6846ec28e65 not found: ID does not exist"
Aug 04 16:06:51.037789 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.037755    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"]
Aug 04 16:06:51.043121 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.043092    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/sub-test-model-4d9499-kserve-5cd496c7b5-g4swm"]
Aug 04 16:06:51.965315 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.965283    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49929c3-2cbe-40da-9a10-f43a995c3964" path="/var/lib/kubelet/pods/d49929c3-2cbe-40da-9a10-f43a995c3964/volumes"
Aug 04 16:06:51.965820 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:06:51.965716    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d60743b2-e049-4369-a608-ed659334d8af" path="/var/lib/kubelet/pods/d60743b2-e049-4369-a608-ed659334d8af/volumes"
Aug 04 16:07:34.796324 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796289    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"]
Aug 04 16:07:34.797010 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796663    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49929c3-2cbe-40da-9a10-f43a995c3964" containerName="main"
Aug 04 16:07:34.797010 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796676    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49929c3-2cbe-40da-9a10-f43a995c3964" containerName="main"
Aug 04 16:07:34.797010 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796693    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d60743b2-e049-4369-a608-ed659334d8af" containerName="main"
Aug 04 16:07:34.797010 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796699    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d60743b2-e049-4369-a608-ed659334d8af" containerName="main"
Aug 04 16:07:34.797010 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796754    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49929c3-2cbe-40da-9a10-f43a995c3964" containerName="main"
Aug 04 16:07:34.797010 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.796763    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d60743b2-e049-4369-a608-ed659334d8af" containerName="main"
Aug 04 16:07:34.799754 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.799734    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:34.802207 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.802187    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"default-dockercfg-wzpgn\""
Aug 04 16:07:34.802504 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.802483    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"kube-root-ca.crt\""
Aug 04 16:07:34.802621 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.802560    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"openshift-service-ca.crt\""
Aug 04 16:07:34.803304 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.803285    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4d9499\"/\"rate-test-model-4d9499-kserve-self-signed-certs\""
Aug 04 16:07:34.810320 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.810296    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"]
Aug 04 16:07:34.904788 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.904753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-model-cache\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:34.904985 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.904813    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-home\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:34.904985 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.904833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-tmp-dir\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:34.904985 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.904904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-dshm\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:34.904985 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.904961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27crk\" (UniqueName: \"kubernetes.io/projected/19c0ff80-8783-435e-bc2b-f0698bee91ed-kube-api-access-27crk\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:34.905134 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:34.905026    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c0ff80-8783-435e-bc2b-f0698bee91ed-tls-certs\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006412 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006361    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-model-cache\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006601 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006446    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-home\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006601 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006471    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-tmp-dir\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006601 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-dshm\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006601 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006554    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27crk\" (UniqueName: \"kubernetes.io/projected/19c0ff80-8783-435e-bc2b-f0698bee91ed-kube-api-access-27crk\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006802 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006616    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c0ff80-8783-435e-bc2b-f0698bee91ed-tls-certs\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006948 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006923    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-model-cache\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.006993 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-home\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.007034 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.006995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-tmp-dir\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.009017 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.008998    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-dshm\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.009284 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.009265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c0ff80-8783-435e-bc2b-f0698bee91ed-tls-certs\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.017424 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.017397    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27crk\" (UniqueName: \"kubernetes.io/projected/19c0ff80-8783-435e-bc2b-f0698bee91ed-kube-api-access-27crk\") pod \"rate-test-model-4d9499-kserve-d78978f4c-lnlgc\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") " pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.109368 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.109254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:35.255789 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:35.255761    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"]
Aug 04 16:07:35.260054 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:07:35.259980    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19c0ff80_8783_435e_bc2b_f0698bee91ed.slice/crio-ef77f3f9ff44a69a02545827e72e34b30b77b34787b9da555679afce01dfc764 WatchSource:0}: Error finding container ef77f3f9ff44a69a02545827e72e34b30b77b34787b9da555679afce01dfc764: Status 404 returned error can't find the container with id ef77f3f9ff44a69a02545827e72e34b30b77b34787b9da555679afce01dfc764
Aug 04 16:07:36.223019 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:36.222976    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc" event={"ID":"19c0ff80-8783-435e-bc2b-f0698bee91ed","Type":"ContainerStarted","Data":"124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3"}
Aug 04 16:07:36.223019 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:36.223026    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc" event={"ID":"19c0ff80-8783-435e-bc2b-f0698bee91ed","Type":"ContainerStarted","Data":"ef77f3f9ff44a69a02545827e72e34b30b77b34787b9da555679afce01dfc764"}
Aug 04 16:07:36.246308 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:36.246252    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc" podStartSLOduration=2.246233501 podStartE2EDuration="2.246233501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:07:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:07:36.244760573 +0000 UTC m=+3398.751058628" watchObservedRunningTime="2026-08-04 16:07:36.246233501 +0000 UTC m=+3398.752531584"
Aug 04 16:07:37.339678 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:37.339623    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:07:45.109828 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:45.109775    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:45.109828 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:45.109835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:45.122724 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:45.122695    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:07:45.272686 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:07:45.272647    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:08:18.937473 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:08:18.937370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:08:27.328148 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:08:27.328106    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:08:59.236540 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:08:59.236504    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:09:05.820550 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:05.820514    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"]
Aug 04 16:09:06.842289 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:06.842251    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-140573164871472"]
Aug 04 16:09:06.845768 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:06.845746    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:06.848325 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:06.848298    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9t7pq\""
Aug 04 16:09:06.853536 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:06.853505    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573164871472"]
Aug 04 16:09:06.942022 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:06.941981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm767\" (UniqueName: \"kubernetes.io/projected/c7be0211-a302-49c3-bfbb-acc7c44f4402-kube-api-access-xm767\") pod \"test-curl-7465-140573164871472\" (UID: \"c7be0211-a302-49c3-bfbb-acc7c44f4402\") " pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:07.042895 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:07.042843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm767\" (UniqueName: \"kubernetes.io/projected/c7be0211-a302-49c3-bfbb-acc7c44f4402-kube-api-access-xm767\") pod \"test-curl-7465-140573164871472\" (UID: \"c7be0211-a302-49c3-bfbb-acc7c44f4402\") " pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:07.052649 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:07.052613    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm767\" (UniqueName: \"kubernetes.io/projected/c7be0211-a302-49c3-bfbb-acc7c44f4402-kube-api-access-xm767\") pod \"test-curl-7465-140573164871472\" (UID: \"c7be0211-a302-49c3-bfbb-acc7c44f4402\") " pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:07.156228 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:07.156186    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:07.302953 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:07.302924    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573164871472"]
Aug 04 16:09:07.305364 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:07.305331    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7be0211_a302_49c3_bfbb_acc7c44f4402.slice/crio-ab151321c4cde121fe4c493f3a93fe9f86d28c69e84e5d9cf80028ab7c99dd34 WatchSource:0}: Error finding container ab151321c4cde121fe4c493f3a93fe9f86d28c69e84e5d9cf80028ab7c99dd34: Status 404 returned error can't find the container with id ab151321c4cde121fe4c493f3a93fe9f86d28c69e84e5d9cf80028ab7c99dd34
Aug 04 16:09:07.306979 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:07.306959    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 16:09:07.655445 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:07.655404    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573164871472" event={"ID":"c7be0211-a302-49c3-bfbb-acc7c44f4402","Type":"ContainerStarted","Data":"ab151321c4cde121fe4c493f3a93fe9f86d28c69e84e5d9cf80028ab7c99dd34"}
Aug 04 16:09:08.661035 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:08.661003    2569 generic.go:356] "Generic (PLEG): container finished" podID="c7be0211-a302-49c3-bfbb-acc7c44f4402" containerID="e9ff3b0473c7a6eafe78d8159351ce81c44dcb541d0855255c6ab8f86a37461e" exitCode=0
Aug 04 16:09:08.661465 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:08.661056    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573164871472" event={"ID":"c7be0211-a302-49c3-bfbb-acc7c44f4402","Type":"ContainerDied","Data":"e9ff3b0473c7a6eafe78d8159351ce81c44dcb541d0855255c6ab8f86a37461e"}
Aug 04 16:09:08.734137 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:08.734099    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:09:09.803420 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:09.803392    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:09.846916 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:09.846848    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7465-140573164871472_c7be0211-a302-49c3-bfbb-acc7c44f4402/test-curl-7465-140573164871472/0.log"
Aug 04 16:09:09.868599 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:09.868557    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-140573164871472"]
Aug 04 16:09:09.875857 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:09.875825    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-140573164871472"]
Aug 04 16:09:09.974094 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:09.973997    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm767\" (UniqueName: \"kubernetes.io/projected/c7be0211-a302-49c3-bfbb-acc7c44f4402-kube-api-access-xm767\") pod \"c7be0211-a302-49c3-bfbb-acc7c44f4402\" (UID: \"c7be0211-a302-49c3-bfbb-acc7c44f4402\") "
Aug 04 16:09:09.976339 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:09.976313    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7be0211-a302-49c3-bfbb-acc7c44f4402-kube-api-access-xm767" (OuterVolumeSpecName: "kube-api-access-xm767") pod "c7be0211-a302-49c3-bfbb-acc7c44f4402" (UID: "c7be0211-a302-49c3-bfbb-acc7c44f4402"). InnerVolumeSpecName "kube-api-access-xm767". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:10.004321 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.004285    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-140573164871728"]
Aug 04 16:09:10.004691 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.004677    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7be0211-a302-49c3-bfbb-acc7c44f4402" containerName="test-curl-7465-140573164871472"
Aug 04 16:09:10.004733 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.004693    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7be0211-a302-49c3-bfbb-acc7c44f4402" containerName="test-curl-7465-140573164871472"
Aug 04 16:09:10.004781 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.004770    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7be0211-a302-49c3-bfbb-acc7c44f4402" containerName="test-curl-7465-140573164871472"
Aug 04 16:09:10.007828 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.007812    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:10.016544 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.016514    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573164871728"]
Aug 04 16:09:10.074695 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.074633    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm767\" (UniqueName: \"kubernetes.io/projected/c7be0211-a302-49c3-bfbb-acc7c44f4402-kube-api-access-xm767\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:10.175599 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.175556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6wbs\" (UniqueName: \"kubernetes.io/projected/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57-kube-api-access-f6wbs\") pod \"test-curl-7465-140573164871728\" (UID: \"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57\") " pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:10.276882 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.276779    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6wbs\" (UniqueName: \"kubernetes.io/projected/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57-kube-api-access-f6wbs\") pod \"test-curl-7465-140573164871728\" (UID: \"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57\") " pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:10.285393 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.285369    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6wbs\" (UniqueName: \"kubernetes.io/projected/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57-kube-api-access-f6wbs\") pod \"test-curl-7465-140573164871728\" (UID: \"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57\") " pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:10.318213 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.318174    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:10.456007 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.455981    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573164871728"]
Aug 04 16:09:10.458665 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:10.458613    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dc4c480_e42b_49b0_92cf_7b3b2ebf7c57.slice/crio-0e4e5021081a054f1323ce34a676f8a23f0ec3131552e89e79f04d5daa4d94d7 WatchSource:0}: Error finding container 0e4e5021081a054f1323ce34a676f8a23f0ec3131552e89e79f04d5daa4d94d7: Status 404 returned error can't find the container with id 0e4e5021081a054f1323ce34a676f8a23f0ec3131552e89e79f04d5daa4d94d7
Aug 04 16:09:10.672179 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.672155    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871472"
Aug 04 16:09:10.672378 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.672201    2569 scope.go:117] "RemoveContainer" containerID="e9ff3b0473c7a6eafe78d8159351ce81c44dcb541d0855255c6ab8f86a37461e"
Aug 04 16:09:10.673712 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:10.673462    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573164871728" event={"ID":"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57","Type":"ContainerStarted","Data":"0e4e5021081a054f1323ce34a676f8a23f0ec3131552e89e79f04d5daa4d94d7"}
Aug 04 16:09:11.684211 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:11.684174    2569 generic.go:356] "Generic (PLEG): container finished" podID="3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" containerID="b9188623cb4c2b3e279801931f57ead4844ca87b95fd34415a613efa0f4a4f92" exitCode=0
Aug 04 16:09:11.684622 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:11.684227    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573164871728" event={"ID":"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57","Type":"ContainerDied","Data":"b9188623cb4c2b3e279801931f57ead4844ca87b95fd34415a613efa0f4a4f92"}
Aug 04 16:09:11.966046 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:11.965960    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7be0211-a302-49c3-bfbb-acc7c44f4402" path="/var/lib/kubelet/pods/c7be0211-a302-49c3-bfbb-acc7c44f4402/volumes"
Aug 04 16:09:12.824832 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:12.824799    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:12.862247 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:12.862218    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7465-140573164871728_3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57/test-curl-7465-140573164871728/0.log"
Aug 04 16:09:12.896715 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:12.896680    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-140573164871728"]
Aug 04 16:09:12.900647 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:12.900607    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-140573164871728"]
Aug 04 16:09:13.003263 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.003149    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6wbs\" (UniqueName: \"kubernetes.io/projected/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57-kube-api-access-f6wbs\") pod \"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57\" (UID: \"3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57\") "
Aug 04 16:09:13.024066 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.024022    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57-kube-api-access-f6wbs" (OuterVolumeSpecName: "kube-api-access-f6wbs") pod "3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" (UID: "3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57"). InnerVolumeSpecName "kube-api-access-f6wbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:13.104514 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.104474    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6wbs\" (UniqueName: \"kubernetes.io/projected/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57-kube-api-access-f6wbs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:13.110216 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.110183    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:13.110578 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.110564    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" containerName="test-curl-7465-140573164871728"
Aug 04 16:09:13.110620 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.110579    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" containerName="test-curl-7465-140573164871728"
Aug 04 16:09:13.110679 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.110655    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" containerName="test-curl-7465-140573164871728"
Aug 04 16:09:13.113552 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.113537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:13.126068 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.126032    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:13.204993 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.204954    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57kw\" (UniqueName: \"kubernetes.io/projected/3448ecdc-29c3-431a-b293-578d8ece05f4-kube-api-access-m57kw\") pod \"test-curl-7465-140573167355200\" (UID: \"3448ecdc-29c3-431a-b293-578d8ece05f4\") " pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:13.306157 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.306050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57kw\" (UniqueName: \"kubernetes.io/projected/3448ecdc-29c3-431a-b293-578d8ece05f4-kube-api-access-m57kw\") pod \"test-curl-7465-140573167355200\" (UID: \"3448ecdc-29c3-431a-b293-578d8ece05f4\") " pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:13.320389 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.320356    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57kw\" (UniqueName: \"kubernetes.io/projected/3448ecdc-29c3-431a-b293-578d8ece05f4-kube-api-access-m57kw\") pod \"test-curl-7465-140573167355200\" (UID: \"3448ecdc-29c3-431a-b293-578d8ece05f4\") " pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:13.423094 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.423056    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:13.560567 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.560485    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:13.563382 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:13.563352    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3448ecdc_29c3_431a_b293_578d8ece05f4.slice/crio-fcb267ab486cf7037c4ed4e08b3b22f83871c72273bf6e8dfa364f119c5388f8 WatchSource:0}: Error finding container fcb267ab486cf7037c4ed4e08b3b22f83871c72273bf6e8dfa364f119c5388f8: Status 404 returned error can't find the container with id fcb267ab486cf7037c4ed4e08b3b22f83871c72273bf6e8dfa364f119c5388f8
Aug 04 16:09:13.695838 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.695807    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:13.696039 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.695847    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e4e5021081a054f1323ce34a676f8a23f0ec3131552e89e79f04d5daa4d94d7"
Aug 04 16:09:13.697058 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.697029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573167355200" event={"ID":"3448ecdc-29c3-431a-b293-578d8ece05f4","Type":"ContainerStarted","Data":"fcb267ab486cf7037c4ed4e08b3b22f83871c72273bf6e8dfa364f119c5388f8"}
Aug 04 16:09:13.707143 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:09:13.707113    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-140573164871728\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" pod="opendatahub/test-curl-7465-140573164871728"
Aug 04 16:09:13.964858 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:13.964822    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57" path="/var/lib/kubelet/pods/3dc4c480-e42b-49b0-92cf-7b3b2ebf7c57/volumes"
Aug 04 16:09:14.702826 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:14.702786    2569 generic.go:356] "Generic (PLEG): container finished" podID="3448ecdc-29c3-431a-b293-578d8ece05f4" containerID="c2537285378b39b605f86092cf37bb84dcdd1e4620c95fdf7f1d81ae43d296bf" exitCode=0
Aug 04 16:09:14.703011 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:14.702870    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573167355200" event={"ID":"3448ecdc-29c3-431a-b293-578d8ece05f4","Type":"ContainerDied","Data":"c2537285378b39b605f86092cf37bb84dcdd1e4620c95fdf7f1d81ae43d296bf"}
Aug 04 16:09:15.845738 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:15.845712    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:15.869590 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:15.869520    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7465-140573167355200_3448ecdc-29c3-431a-b293-578d8ece05f4/test-curl-7465-140573167355200/0.log"
Aug 04 16:09:15.898320 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:15.898281    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:15.902276 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:15.902247    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:16.015866 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.015783    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:16.016138 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.016125    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3448ecdc-29c3-431a-b293-578d8ece05f4" containerName="test-curl-7465-140573167355200"
Aug 04 16:09:16.016178 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.016140    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3448ecdc-29c3-431a-b293-578d8ece05f4" containerName="test-curl-7465-140573167355200"
Aug 04 16:09:16.016258 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.016207    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3448ecdc-29c3-431a-b293-578d8ece05f4" containerName="test-curl-7465-140573167355200"
Aug 04 16:09:16.018990 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.018974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:16.027027 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.026995    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:16.029827 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.029802    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57kw\" (UniqueName: \"kubernetes.io/projected/3448ecdc-29c3-431a-b293-578d8ece05f4-kube-api-access-m57kw\") pod \"3448ecdc-29c3-431a-b293-578d8ece05f4\" (UID: \"3448ecdc-29c3-431a-b293-578d8ece05f4\") "
Aug 04 16:09:16.032354 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.032321    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3448ecdc-29c3-431a-b293-578d8ece05f4-kube-api-access-m57kw" (OuterVolumeSpecName: "kube-api-access-m57kw") pod "3448ecdc-29c3-431a-b293-578d8ece05f4" (UID: "3448ecdc-29c3-431a-b293-578d8ece05f4"). InnerVolumeSpecName "kube-api-access-m57kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:16.130693 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.130656    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl47p\" (UniqueName: \"kubernetes.io/projected/cc3749ae-0446-4f7e-ad6d-bec048183097-kube-api-access-fl47p\") pod \"test-curl-7465-140573167355200\" (UID: \"cc3749ae-0446-4f7e-ad6d-bec048183097\") " pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:16.130892 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.130781    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57kw\" (UniqueName: \"kubernetes.io/projected/3448ecdc-29c3-431a-b293-578d8ece05f4-kube-api-access-m57kw\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:16.231557 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.231515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl47p\" (UniqueName: \"kubernetes.io/projected/cc3749ae-0446-4f7e-ad6d-bec048183097-kube-api-access-fl47p\") pod \"test-curl-7465-140573167355200\" (UID: \"cc3749ae-0446-4f7e-ad6d-bec048183097\") " pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:16.241246 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.241214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl47p\" (UniqueName: \"kubernetes.io/projected/cc3749ae-0446-4f7e-ad6d-bec048183097-kube-api-access-fl47p\") pod \"test-curl-7465-140573167355200\" (UID: \"cc3749ae-0446-4f7e-ad6d-bec048183097\") " pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:16.329148 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.329063    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:16.469941 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.469905    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:16.473290 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:16.473259    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc3749ae_0446_4f7e_ad6d_bec048183097.slice/crio-0778b4609614f38b2b08bec682e9dd6e3d80e391f7825ab895f0d92842861d60 WatchSource:0}: Error finding container 0778b4609614f38b2b08bec682e9dd6e3d80e391f7825ab895f0d92842861d60: Status 404 returned error can't find the container with id 0778b4609614f38b2b08bec682e9dd6e3d80e391f7825ab895f0d92842861d60
Aug 04 16:09:16.715878 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.715846    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:16.716087 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.715890    2569 scope.go:117] "RemoveContainer" containerID="c2537285378b39b605f86092cf37bb84dcdd1e4620c95fdf7f1d81ae43d296bf"
Aug 04 16:09:16.717308 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:16.717112    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573167355200" event={"ID":"cc3749ae-0446-4f7e-ad6d-bec048183097","Type":"ContainerStarted","Data":"0778b4609614f38b2b08bec682e9dd6e3d80e391f7825ab895f0d92842861d60"}
Aug 04 16:09:17.723258 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:17.723218    2569 generic.go:356] "Generic (PLEG): container finished" podID="cc3749ae-0446-4f7e-ad6d-bec048183097" containerID="71143aacc0db96f9eb385aa590cc1d8dad2e42bf584c9179ccfdaff6ef746374" exitCode=0
Aug 04 16:09:17.723745 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:17.723303    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573167355200" event={"ID":"cc3749ae-0446-4f7e-ad6d-bec048183097","Type":"ContainerDied","Data":"71143aacc0db96f9eb385aa590cc1d8dad2e42bf584c9179ccfdaff6ef746374"}
Aug 04 16:09:17.966499 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:17.966461    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3448ecdc-29c3-431a-b293-578d8ece05f4" path="/var/lib/kubelet/pods/3448ecdc-29c3-431a-b293-578d8ece05f4/volumes"
Aug 04 16:09:18.872938 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:18.872909    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:18.898233 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:18.898201    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7465-140573167355200_cc3749ae-0446-4f7e-ad6d-bec048183097/test-curl-7465-140573167355200/0.log"
Aug 04 16:09:18.930992 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:18.930954    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:18.941490 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:18.941457    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-140573167355200"]
Aug 04 16:09:18.957896 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:18.957863    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fl47p\" (UniqueName: \"kubernetes.io/projected/cc3749ae-0446-4f7e-ad6d-bec048183097-kube-api-access-fl47p\") pod \"cc3749ae-0446-4f7e-ad6d-bec048183097\" (UID: \"cc3749ae-0446-4f7e-ad6d-bec048183097\") "
Aug 04 16:09:18.960005 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:18.959982    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc3749ae-0446-4f7e-ad6d-bec048183097-kube-api-access-fl47p" (OuterVolumeSpecName: "kube-api-access-fl47p") pod "cc3749ae-0446-4f7e-ad6d-bec048183097" (UID: "cc3749ae-0446-4f7e-ad6d-bec048183097"). InnerVolumeSpecName "kube-api-access-fl47p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:19.058889 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.058798    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl47p\" (UniqueName: \"kubernetes.io/projected/cc3749ae-0446-4f7e-ad6d-bec048183097-kube-api-access-fl47p\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:19.074849 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.074810    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:19.075232 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.075218    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc3749ae-0446-4f7e-ad6d-bec048183097" containerName="test-curl-7465-140573167355200"
Aug 04 16:09:19.075279 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.075234    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc3749ae-0446-4f7e-ad6d-bec048183097" containerName="test-curl-7465-140573167355200"
Aug 04 16:09:19.075316 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.075309    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc3749ae-0446-4f7e-ad6d-bec048183097" containerName="test-curl-7465-140573167355200"
Aug 04 16:09:19.078341 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.078323    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:19.086969 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.086937    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:19.159776 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.159738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlmdw\" (UniqueName: \"kubernetes.io/projected/cb84c654-ee72-4723-8152-34a9086d3436-kube-api-access-wlmdw\") pod \"test-curl-7465-140573165009488\" (UID: \"cb84c654-ee72-4723-8152-34a9086d3436\") " pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:19.260918 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.260876    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlmdw\" (UniqueName: \"kubernetes.io/projected/cb84c654-ee72-4723-8152-34a9086d3436-kube-api-access-wlmdw\") pod \"test-curl-7465-140573165009488\" (UID: \"cb84c654-ee72-4723-8152-34a9086d3436\") " pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:19.269995 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.269959    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlmdw\" (UniqueName: \"kubernetes.io/projected/cb84c654-ee72-4723-8152-34a9086d3436-kube-api-access-wlmdw\") pod \"test-curl-7465-140573165009488\" (UID: \"cb84c654-ee72-4723-8152-34a9086d3436\") " pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:19.388077 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.388045    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:19.527561 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.527533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:19.529958 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:19.529929    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb84c654_ee72_4723_8152_34a9086d3436.slice/crio-7404a4c5eebb02c2ce9ca9cbda5c13beb9185a5360da18a2833a8b1a59162e9d WatchSource:0}: Error finding container 7404a4c5eebb02c2ce9ca9cbda5c13beb9185a5360da18a2833a8b1a59162e9d: Status 404 returned error can't find the container with id 7404a4c5eebb02c2ce9ca9cbda5c13beb9185a5360da18a2833a8b1a59162e9d
Aug 04 16:09:19.735826 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.735735    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:19.735826 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.735784    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0778b4609614f38b2b08bec682e9dd6e3d80e391f7825ab895f0d92842861d60"
Aug 04 16:09:19.737028 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.736995    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573165009488" event={"ID":"cb84c654-ee72-4723-8152-34a9086d3436","Type":"ContainerStarted","Data":"7404a4c5eebb02c2ce9ca9cbda5c13beb9185a5360da18a2833a8b1a59162e9d"}
Aug 04 16:09:19.747123 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:09:19.747096    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-140573167355200\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="cc3749ae-0446-4f7e-ad6d-bec048183097" pod="opendatahub/test-curl-7465-140573167355200"
Aug 04 16:09:19.965392 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:19.965359    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc3749ae-0446-4f7e-ad6d-bec048183097" path="/var/lib/kubelet/pods/cc3749ae-0446-4f7e-ad6d-bec048183097/volumes"
Aug 04 16:09:20.743566 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:20.743533    2569 generic.go:356] "Generic (PLEG): container finished" podID="cb84c654-ee72-4723-8152-34a9086d3436" containerID="132325d1cc6ffd1d45c1850a303d7353d362543cb49f290e9a5b6ec89242f288" exitCode=0
Aug 04 16:09:20.743809 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:20.743618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573165009488" event={"ID":"cb84c654-ee72-4723-8152-34a9086d3436","Type":"ContainerDied","Data":"132325d1cc6ffd1d45c1850a303d7353d362543cb49f290e9a5b6ec89242f288"}
Aug 04 16:09:20.839699 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:20.839659    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc" podUID="19c0ff80-8783-435e-bc2b-f0698bee91ed" containerName="main" containerID="cri-o://124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3" gracePeriod=45
Aug 04 16:09:21.099190 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.099160    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:09:21.178162 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178123    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-dshm\") pod \"19c0ff80-8783-435e-bc2b-f0698bee91ed\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") "
Aug 04 16:09:21.178362 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178179    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c0ff80-8783-435e-bc2b-f0698bee91ed-tls-certs\") pod \"19c0ff80-8783-435e-bc2b-f0698bee91ed\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") "
Aug 04 16:09:21.178362 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178237    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-home\") pod \"19c0ff80-8783-435e-bc2b-f0698bee91ed\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") "
Aug 04 16:09:21.178362 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178265    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-tmp-dir\") pod \"19c0ff80-8783-435e-bc2b-f0698bee91ed\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") "
Aug 04 16:09:21.178362 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178296    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-model-cache\") pod \"19c0ff80-8783-435e-bc2b-f0698bee91ed\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") "
Aug 04 16:09:21.178362 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178324    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27crk\" (UniqueName: \"kubernetes.io/projected/19c0ff80-8783-435e-bc2b-f0698bee91ed-kube-api-access-27crk\") pod \"19c0ff80-8783-435e-bc2b-f0698bee91ed\" (UID: \"19c0ff80-8783-435e-bc2b-f0698bee91ed\") "
Aug 04 16:09:21.178718 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178496    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-home" (OuterVolumeSpecName: "home") pod "19c0ff80-8783-435e-bc2b-f0698bee91ed" (UID: "19c0ff80-8783-435e-bc2b-f0698bee91ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:09:21.178718 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178531    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19c0ff80-8783-435e-bc2b-f0698bee91ed" (UID: "19c0ff80-8783-435e-bc2b-f0698bee91ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:09:21.178835 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178730    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "19c0ff80-8783-435e-bc2b-f0698bee91ed" (UID: "19c0ff80-8783-435e-bc2b-f0698bee91ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:09:21.178835 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178819    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:21.178914 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.178839    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:21.180463 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.180445    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-dshm" (OuterVolumeSpecName: "dshm") pod "19c0ff80-8783-435e-bc2b-f0698bee91ed" (UID: "19c0ff80-8783-435e-bc2b-f0698bee91ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:09:21.180608 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.180586    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19c0ff80-8783-435e-bc2b-f0698bee91ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19c0ff80-8783-435e-bc2b-f0698bee91ed" (UID: "19c0ff80-8783-435e-bc2b-f0698bee91ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 16:09:21.180869 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.180846    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19c0ff80-8783-435e-bc2b-f0698bee91ed-kube-api-access-27crk" (OuterVolumeSpecName: "kube-api-access-27crk") pod "19c0ff80-8783-435e-bc2b-f0698bee91ed" (UID: "19c0ff80-8783-435e-bc2b-f0698bee91ed"). InnerVolumeSpecName "kube-api-access-27crk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:21.280349 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.280261    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:21.280349 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.280293    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27crk\" (UniqueName: \"kubernetes.io/projected/19c0ff80-8783-435e-bc2b-f0698bee91ed-kube-api-access-27crk\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:21.280349 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.280304    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19c0ff80-8783-435e-bc2b-f0698bee91ed-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:21.280349 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.280314    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19c0ff80-8783-435e-bc2b-f0698bee91ed-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:21.749704 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.749668    2569 generic.go:356] "Generic (PLEG): container finished" podID="19c0ff80-8783-435e-bc2b-f0698bee91ed" containerID="124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3" exitCode=0
Aug 04 16:09:21.749892 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.749768    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"
Aug 04 16:09:21.749892 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.749802    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc" event={"ID":"19c0ff80-8783-435e-bc2b-f0698bee91ed","Type":"ContainerDied","Data":"124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3"}
Aug 04 16:09:21.749892 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.749844    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc" event={"ID":"19c0ff80-8783-435e-bc2b-f0698bee91ed","Type":"ContainerDied","Data":"ef77f3f9ff44a69a02545827e72e34b30b77b34787b9da555679afce01dfc764"}
Aug 04 16:09:21.749892 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.749864    2569 scope.go:117] "RemoveContainer" containerID="124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3"
Aug 04 16:09:21.759718 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.759694    2569 scope.go:117] "RemoveContainer" containerID="124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3"
Aug 04 16:09:21.760036 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:09:21.760015    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3\": container with ID starting with 124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3 not found: ID does not exist" containerID="124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3"
Aug 04 16:09:21.760095 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.760047    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3"} err="failed to get container status \"124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3\": rpc error: code = NotFound desc = could not find container \"124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3\": container with ID starting with 124af1a6efa99c88efd4b7fee476e3ab9cc4e43e8576cde7d088536201d9bde3 not found: ID does not exist"
Aug 04 16:09:21.775506 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.775466    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"]
Aug 04 16:09:21.777208 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.777179    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4d9499/rate-test-model-4d9499-kserve-d78978f4c-lnlgc"]
Aug 04 16:09:21.883847 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.883817    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:21.908267 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.908236    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7465-140573165009488_cb84c654-ee72-4723-8152-34a9086d3436/test-curl-7465-140573165009488/0.log"
Aug 04 16:09:21.932974 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.932935    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:21.937977 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.937945    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:21.968191 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.968152    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19c0ff80-8783-435e-bc2b-f0698bee91ed" path="/var/lib/kubelet/pods/19c0ff80-8783-435e-bc2b-f0698bee91ed/volumes"
Aug 04 16:09:21.987556 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.987519    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlmdw\" (UniqueName: \"kubernetes.io/projected/cb84c654-ee72-4723-8152-34a9086d3436-kube-api-access-wlmdw\") pod \"cb84c654-ee72-4723-8152-34a9086d3436\" (UID: \"cb84c654-ee72-4723-8152-34a9086d3436\") "
Aug 04 16:09:21.989859 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:21.989831    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb84c654-ee72-4723-8152-34a9086d3436-kube-api-access-wlmdw" (OuterVolumeSpecName: "kube-api-access-wlmdw") pod "cb84c654-ee72-4723-8152-34a9086d3436" (UID: "cb84c654-ee72-4723-8152-34a9086d3436"). InnerVolumeSpecName "kube-api-access-wlmdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:22.069753 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.069662    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:22.070064 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.070051    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19c0ff80-8783-435e-bc2b-f0698bee91ed" containerName="main"
Aug 04 16:09:22.070116 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.070065    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="19c0ff80-8783-435e-bc2b-f0698bee91ed" containerName="main"
Aug 04 16:09:22.070116 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.070076    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb84c654-ee72-4723-8152-34a9086d3436" containerName="test-curl-7465-140573165009488"
Aug 04 16:09:22.070116 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.070083    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb84c654-ee72-4723-8152-34a9086d3436" containerName="test-curl-7465-140573165009488"
Aug 04 16:09:22.070227 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.070148    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb84c654-ee72-4723-8152-34a9086d3436" containerName="test-curl-7465-140573165009488"
Aug 04 16:09:22.070227 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.070157    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="19c0ff80-8783-435e-bc2b-f0698bee91ed" containerName="main"
Aug 04 16:09:22.073697 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.073674    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:22.081362 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.081327    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:22.088370 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.088264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfl8\" (UniqueName: \"kubernetes.io/projected/4063b6f8-6374-46ca-b5db-fed3b1fda518-kube-api-access-2kfl8\") pod \"test-curl-7465-140573165009488\" (UID: \"4063b6f8-6374-46ca-b5db-fed3b1fda518\") " pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:22.088557 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.088521    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlmdw\" (UniqueName: \"kubernetes.io/projected/cb84c654-ee72-4723-8152-34a9086d3436-kube-api-access-wlmdw\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:22.189013 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.188977    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfl8\" (UniqueName: \"kubernetes.io/projected/4063b6f8-6374-46ca-b5db-fed3b1fda518-kube-api-access-2kfl8\") pod \"test-curl-7465-140573165009488\" (UID: \"4063b6f8-6374-46ca-b5db-fed3b1fda518\") " pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:22.197878 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.197854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfl8\" (UniqueName: \"kubernetes.io/projected/4063b6f8-6374-46ca-b5db-fed3b1fda518-kube-api-access-2kfl8\") pod \"test-curl-7465-140573165009488\" (UID: \"4063b6f8-6374-46ca-b5db-fed3b1fda518\") " pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:22.391279 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.391240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:22.536920 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.533519    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:22.757754 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.757629    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:22.757754 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.757676    2569 scope.go:117] "RemoveContainer" containerID="132325d1cc6ffd1d45c1850a303d7353d362543cb49f290e9a5b6ec89242f288"
Aug 04 16:09:22.758864 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:22.758826    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573165009488" event={"ID":"4063b6f8-6374-46ca-b5db-fed3b1fda518","Type":"ContainerStarted","Data":"66ea0eff8a559fff9e3478cc65dd09e1618f583564016b7a4026947392461090"}
Aug 04 16:09:23.766202 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:23.766161    2569 generic.go:356] "Generic (PLEG): container finished" podID="4063b6f8-6374-46ca-b5db-fed3b1fda518" containerID="2c0a821359c38538f08aac364d6fab07cf2cdb38777e417301f68de96f4d8e26" exitCode=0
Aug 04 16:09:23.766606 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:23.766241    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7465-140573165009488" event={"ID":"4063b6f8-6374-46ca-b5db-fed3b1fda518","Type":"ContainerDied","Data":"2c0a821359c38538f08aac364d6fab07cf2cdb38777e417301f68de96f4d8e26"}
Aug 04 16:09:23.965405 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:23.965368    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb84c654-ee72-4723-8152-34a9086d3436" path="/var/lib/kubelet/pods/cb84c654-ee72-4723-8152-34a9086d3436/volumes"
Aug 04 16:09:24.911707 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:24.911678    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:24.934011 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:24.933964    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7465-140573165009488_4063b6f8-6374-46ca-b5db-fed3b1fda518/test-curl-7465-140573165009488/0.log"
Aug 04 16:09:24.955887 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:24.955853    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:24.958145 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:24.958108    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7465-140573165009488"]
Aug 04 16:09:25.017025 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:25.016983    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kfl8\" (UniqueName: \"kubernetes.io/projected/4063b6f8-6374-46ca-b5db-fed3b1fda518-kube-api-access-2kfl8\") pod \"4063b6f8-6374-46ca-b5db-fed3b1fda518\" (UID: \"4063b6f8-6374-46ca-b5db-fed3b1fda518\") "
Aug 04 16:09:25.019127 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:25.019102    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4063b6f8-6374-46ca-b5db-fed3b1fda518-kube-api-access-2kfl8" (OuterVolumeSpecName: "kube-api-access-2kfl8") pod "4063b6f8-6374-46ca-b5db-fed3b1fda518" (UID: "4063b6f8-6374-46ca-b5db-fed3b1fda518"). InnerVolumeSpecName "kube-api-access-2kfl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:09:25.118030 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:25.117998    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kfl8\" (UniqueName: \"kubernetes.io/projected/4063b6f8-6374-46ca-b5db-fed3b1fda518-kube-api-access-2kfl8\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:09:25.777768 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:25.777736    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:25.777947 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:25.777774    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66ea0eff8a559fff9e3478cc65dd09e1618f583564016b7a4026947392461090"
Aug 04 16:09:25.798886 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:09:25.798808    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7465-140573165009488\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="4063b6f8-6374-46ca-b5db-fed3b1fda518" pod="opendatahub/test-curl-7465-140573165009488"
Aug 04 16:09:25.965031 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:25.964988    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4063b6f8-6374-46ca-b5db-fed3b1fda518" path="/var/lib/kubelet/pods/4063b6f8-6374-46ca-b5db-fed3b1fda518/volumes"
Aug 04 16:09:31.423332 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.423291    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"]
Aug 04 16:09:31.423778 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.423678    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4063b6f8-6374-46ca-b5db-fed3b1fda518" containerName="test-curl-7465-140573165009488"
Aug 04 16:09:31.423778 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.423692    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4063b6f8-6374-46ca-b5db-fed3b1fda518" containerName="test-curl-7465-140573165009488"
Aug 04 16:09:31.423778 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.423763    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4063b6f8-6374-46ca-b5db-fed3b1fda518" containerName="test-curl-7465-140573165009488"
Aug 04 16:09:31.426781 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.426757    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.430477 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.430453    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8ff9cf\""
Aug 04 16:09:31.430599 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.430479    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8ff9cf-dockercfg-hm569\""
Aug 04 16:09:31.442482 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.442453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"]
Aug 04 16:09:31.461959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.461929    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"]
Aug 04 16:09:31.465339 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.465317    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.477155 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.477129    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.477301 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.477177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5bwk\" (UniqueName: \"kubernetes.io/projected/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-kube-api-access-t5bwk\") pod \"payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.478894 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.478871    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"]
Aug 04 16:09:31.578765 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.578729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.578959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.578775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5bwk\" (UniqueName: \"kubernetes.io/projected/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-kube-api-access-t5bwk\") pod \"payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.578959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.578816    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvl2v\" (UniqueName: \"kubernetes.io/projected/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-kube-api-access-mvl2v\") pod \"payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.578959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.578877    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.579450 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.579430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.588476 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.588433    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5bwk\" (UniqueName: \"kubernetes.io/projected/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-kube-api-access-t5bwk\") pod \"payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.679526 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.679430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvl2v\" (UniqueName: \"kubernetes.io/projected/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-kube-api-access-mvl2v\") pod \"payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.679728 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.679612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.680286 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.680268    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.688897 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.688871    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvl2v\" (UniqueName: \"kubernetes.io/projected/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-kube-api-access-mvl2v\") pod \"payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.737366 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.737327    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:31.776037 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.776003    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:31.901098 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:31.901057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"]
Aug 04 16:09:31.905343 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:31.905307    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce527452_4ed5_4cfd_b1a1_e289ae2b6b9a.slice/crio-fd06db053141f1f76dca07bb63aa33e2774c90a6be63ff9cafbbbd845e72c595 WatchSource:0}: Error finding container fd06db053141f1f76dca07bb63aa33e2774c90a6be63ff9cafbbbd845e72c595: Status 404 returned error can't find the container with id fd06db053141f1f76dca07bb63aa33e2774c90a6be63ff9cafbbbd845e72c595
Aug 04 16:09:32.150494 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.150466    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"]
Aug 04 16:09:32.153141 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:09:32.153098    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80c4390_2b1a_4daa_8735_7ce421a9d7c9.slice/crio-5d12bbdc4ea49caa2a51d5b56b2ccfb5c08ce5c85c7a93f731e4e55dc0ec40f4 WatchSource:0}: Error finding container 5d12bbdc4ea49caa2a51d5b56b2ccfb5c08ce5c85c7a93f731e4e55dc0ec40f4: Status 404 returned error can't find the container with id 5d12bbdc4ea49caa2a51d5b56b2ccfb5c08ce5c85c7a93f731e4e55dc0ec40f4
Aug 04 16:09:32.820718 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.820682    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn" event={"ID":"d80c4390-2b1a-4daa-8735-7ce421a9d7c9","Type":"ContainerStarted","Data":"b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4"}
Aug 04 16:09:32.821157 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.820741    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn" event={"ID":"d80c4390-2b1a-4daa-8735-7ce421a9d7c9","Type":"ContainerStarted","Data":"5d12bbdc4ea49caa2a51d5b56b2ccfb5c08ce5c85c7a93f731e4e55dc0ec40f4"}
Aug 04 16:09:32.821157 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.820773    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:32.822269 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.822243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4" event={"ID":"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a","Type":"ContainerStarted","Data":"8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530"}
Aug 04 16:09:32.822396 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.822286    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4" event={"ID":"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a","Type":"ContainerStarted","Data":"fd06db053141f1f76dca07bb63aa33e2774c90a6be63ff9cafbbbd845e72c595"}
Aug 04 16:09:32.822396 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.822343    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:09:32.842974 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.842916    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn" podStartSLOduration=1.842894888 podStartE2EDuration="1.842894888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:32.841015065 +0000 UTC m=+3515.347313120" watchObservedRunningTime="2026-08-04 16:09:32.842894888 +0000 UTC m=+3515.349192944"
Aug 04 16:09:32.861130 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:32.861079    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4" podStartSLOduration=1.861057988 podStartE2EDuration="1.861057988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:09:32.860046353 +0000 UTC m=+3515.366344402" watchObservedRunningTime="2026-08-04 16:09:32.861057988 +0000 UTC m=+3515.367356047"
Aug 04 16:09:34.135909 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:34.135872    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:09:43.829596 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:43.829551    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:09:43.830099 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:09:43.829699    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:10:00.466654 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.466605    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"]
Aug 04 16:10:00.472271 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.472245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.475620 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.475594    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2b50f0-dockercfg-79jgn\""
Aug 04 16:10:00.476104 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.476083    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2b50f0\""
Aug 04 16:10:00.493793 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.493756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"]
Aug 04 16:10:00.543715 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.543599    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbqr6\" (UniqueName: \"kubernetes.io/projected/56ff87fc-bba4-4f53-9998-04329208c002-kube-api-access-rbqr6\") pod \"payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.543715 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.543676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ff87fc-bba4-4f53-9998-04329208c002-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.556826 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.556792    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"]
Aug 04 16:10:00.560425 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.560408    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.587602 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.582688    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"]
Aug 04 16:10:00.644293 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.644253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.644497 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.644318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vxsm\" (UniqueName: \"kubernetes.io/projected/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-kube-api-access-8vxsm\") pod \"payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.644497 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.644350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbqr6\" (UniqueName: \"kubernetes.io/projected/56ff87fc-bba4-4f53-9998-04329208c002-kube-api-access-rbqr6\") pod \"payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.644497 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.644374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ff87fc-bba4-4f53-9998-04329208c002-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.645147 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.645097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ff87fc-bba4-4f53-9998-04329208c002-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.654401 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.654364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbqr6\" (UniqueName: \"kubernetes.io/projected/56ff87fc-bba4-4f53-9998-04329208c002-kube-api-access-rbqr6\") pod \"payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.745415 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.745318    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vxsm\" (UniqueName: \"kubernetes.io/projected/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-kube-api-access-8vxsm\") pod \"payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.745415 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.745415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.746182 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.746164    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.761978 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.761944    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vxsm\" (UniqueName: \"kubernetes.io/projected/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-kube-api-access-8vxsm\") pod \"payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.783000 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.782959    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:00.870975 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.870938    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:00.938686 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.938390    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"]
Aug 04 16:10:00.942088 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:10:00.942046    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ff87fc_bba4_4f53_9998_04329208c002.slice/crio-17b1b38fdc197961e840d97bb7c879328740ff08f630349e4eed1cc098e4c114 WatchSource:0}: Error finding container 17b1b38fdc197961e840d97bb7c879328740ff08f630349e4eed1cc098e4c114: Status 404 returned error can't find the container with id 17b1b38fdc197961e840d97bb7c879328740ff08f630349e4eed1cc098e4c114
Aug 04 16:10:00.970984 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:00.970947    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs" event={"ID":"56ff87fc-bba4-4f53-9998-04329208c002","Type":"ContainerStarted","Data":"17b1b38fdc197961e840d97bb7c879328740ff08f630349e4eed1cc098e4c114"}
Aug 04 16:10:01.040664 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.040620    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"]
Aug 04 16:10:01.042422 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:10:01.042391    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02fa956f_4692_44bf_ac25_ee1ddb3cfde8.slice/crio-1a0357028c555d6b5f797dee42e1d77fafc14c9521891fe3096b4d1fd5dc1f79 WatchSource:0}: Error finding container 1a0357028c555d6b5f797dee42e1d77fafc14c9521891fe3096b4d1fd5dc1f79: Status 404 returned error can't find the container with id 1a0357028c555d6b5f797dee42e1d77fafc14c9521891fe3096b4d1fd5dc1f79
Aug 04 16:10:01.978323 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.978284    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs" event={"ID":"56ff87fc-bba4-4f53-9998-04329208c002","Type":"ContainerStarted","Data":"ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044"}
Aug 04 16:10:01.978883 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.978354    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:01.979799 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.979773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5" event={"ID":"02fa956f-4692-44bf-ac25-ee1ddb3cfde8","Type":"ContainerStarted","Data":"f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c"}
Aug 04 16:10:01.979920 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.979822    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5" event={"ID":"02fa956f-4692-44bf-ac25-ee1ddb3cfde8","Type":"ContainerStarted","Data":"1a0357028c555d6b5f797dee42e1d77fafc14c9521891fe3096b4d1fd5dc1f79"}
Aug 04 16:10:01.979920 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.979860    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:01.996263 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:01.996207    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs" podStartSLOduration=1.99618944 podStartE2EDuration="1.99618944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:10:01.995511479 +0000 UTC m=+3544.501809554" watchObservedRunningTime="2026-08-04 16:10:01.99618944 +0000 UTC m=+3544.502487495"
Aug 04 16:10:02.013459 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:02.013400    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5" podStartSLOduration=2.013383522 podStartE2EDuration="2.013383522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:10:02.011761951 +0000 UTC m=+3544.518060045" watchObservedRunningTime="2026-08-04 16:10:02.013383522 +0000 UTC m=+3544.519681577"
Aug 04 16:10:03.514430 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:03.514382    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:10:12.988096 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:12.988064    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:10:12.988485 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:12.988148    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:10:30.583754 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:30.583717    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5_02fa956f-4692-44bf-ac25-ee1ddb3cfde8/payload-processing/0.log"
Aug 04 16:10:45.297238 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.297203    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"]
Aug 04 16:10:45.300550 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.300532    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.304232 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.304207    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8ff9cf\"/\"default-dockercfg-cfdf8\""
Aug 04 16:10:45.304232 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.304225    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8ff9cf\"/\"kube-root-ca.crt\""
Aug 04 16:10:45.304431 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.304225    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-8ff9cf\"/\"openshift-service-ca.crt\""
Aug 04 16:10:45.304431 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.304375    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-8ff9cf\"/\"ipp-route-8ff9cf-kserve-self-signed-certs\""
Aug 04 16:10:45.313407 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.313380    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"]
Aug 04 16:10:45.355313 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.355270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-model-cache\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.355518 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.355348    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6587d644-e186-4878-bd6c-38f37c99e5bc-tls-certs\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.355518 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.355418    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qqxw\" (UniqueName: \"kubernetes.io/projected/6587d644-e186-4878-bd6c-38f37c99e5bc-kube-api-access-8qqxw\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.355518 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.355457    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-home\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.355518 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.355487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-tmp-dir\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.355835 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.355590    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-dshm\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457125 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-home\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457125 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-tmp-dir\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457349 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-dshm\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457349 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-model-cache\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457422 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6587d644-e186-4878-bd6c-38f37c99e5bc-tls-certs\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457464 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qqxw\" (UniqueName: \"kubernetes.io/projected/6587d644-e186-4878-bd6c-38f37c99e5bc-kube-api-access-8qqxw\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457536 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-tmp-dir\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457602 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-home\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.457688 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.457669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-model-cache\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.459527 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.459501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-dshm\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.459960 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.459941    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6587d644-e186-4878-bd6c-38f37c99e5bc-tls-certs\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.465984 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.465955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qqxw\" (UniqueName: \"kubernetes.io/projected/6587d644-e186-4878-bd6c-38f37c99e5bc-kube-api-access-8qqxw\") pod \"ipp-route-8ff9cf-kserve-766bc69747-t7br2\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") " pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.610904 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.610808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:45.754352 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:45.754327    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"]
Aug 04 16:10:45.756506 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:10:45.756478    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6587d644_e186_4878_bd6c_38f37c99e5bc.slice/crio-710c3954dba1aeecdb42373edd4c3a4ede171e2a5ebc34117640eac3c1d31c7e WatchSource:0}: Error finding container 710c3954dba1aeecdb42373edd4c3a4ede171e2a5ebc34117640eac3c1d31c7e: Status 404 returned error can't find the container with id 710c3954dba1aeecdb42373edd4c3a4ede171e2a5ebc34117640eac3c1d31c7e
Aug 04 16:10:46.201097 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:46.201052    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2" event={"ID":"6587d644-e186-4878-bd6c-38f37c99e5bc","Type":"ContainerStarted","Data":"04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52"}
Aug 04 16:10:46.201097 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:46.201095    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2" event={"ID":"6587d644-e186-4878-bd6c-38f37c99e5bc","Type":"ContainerStarted","Data":"710c3954dba1aeecdb42373edd4c3a4ede171e2a5ebc34117640eac3c1d31c7e"}
Aug 04 16:10:46.221809 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:46.221750    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2" podStartSLOduration=1.221733835 podStartE2EDuration="1.221733835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:10:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:10:46.219845995 +0000 UTC m=+3588.726144042" watchObservedRunningTime="2026-08-04 16:10:46.221733835 +0000 UTC m=+3588.728031951"
Aug 04 16:10:48.018874 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:48.018836    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:10:55.611771 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:55.611677    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:55.611771 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:55.611736    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:55.624415 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:55.624383    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:10:56.262725 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:10:56.262695    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:11:14.513573 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:14.513527    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:11:26.556386 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:26.556351    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4_ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a/payload-processing/0.log"
Aug 04 16:11:26.821605 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:26.821512    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5_02fa956f-4692-44bf-ac25-ee1ddb3cfde8/payload-processing/0.log"
Aug 04 16:11:31.892169 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.888906    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"]
Aug 04 16:11:31.894663 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.894600    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:31.898120 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.898095    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8528c9-dockercfg-qj6st\""
Aug 04 16:11:31.898253 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.898095    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8528c9\""
Aug 04 16:11:31.913331 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.913299    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"]
Aug 04 16:11:31.917850 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.917823    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"]
Aug 04 16:11:31.921392 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.921374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:31.958400 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:31.958363    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"]
Aug 04 16:11:32.005958 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.005885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df7xl\" (UniqueName: \"kubernetes.io/projected/19996ec8-4290-4e5a-8b24-3e5afcf20642-kube-api-access-df7xl\") pod \"payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.005958 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.005935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wjl\" (UniqueName: \"kubernetes.io/projected/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-kube-api-access-72wjl\") pod \"payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.006240 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.006050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19996ec8-4290-4e5a-8b24-3e5afcf20642-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.006240 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.006090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.107014 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.106967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df7xl\" (UniqueName: \"kubernetes.io/projected/19996ec8-4290-4e5a-8b24-3e5afcf20642-kube-api-access-df7xl\") pod \"payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.107014 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.107010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wjl\" (UniqueName: \"kubernetes.io/projected/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-kube-api-access-72wjl\") pod \"payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.107309 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.107049    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19996ec8-4290-4e5a-8b24-3e5afcf20642-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.107309 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.107066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.108120 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.108093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19996ec8-4290-4e5a-8b24-3e5afcf20642-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.108173 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.108093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.120187 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.120150    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wjl\" (UniqueName: \"kubernetes.io/projected/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-kube-api-access-72wjl\") pod \"payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.120354 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.120152    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df7xl\" (UniqueName: \"kubernetes.io/projected/19996ec8-4290-4e5a-8b24-3e5afcf20642-kube-api-access-df7xl\") pod \"payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.206590 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.206500    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:32.234894 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.234859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:32.371715 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.371678    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"]
Aug 04 16:11:32.376631 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:11:32.376601    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bbb7015_bcc6_43fd_8e85_5e80edc6f8e5.slice/crio-90e5a7aa85a83f16b2c1c3e02f28d6d1d066ede751034f69e2043b8207e515a3 WatchSource:0}: Error finding container 90e5a7aa85a83f16b2c1c3e02f28d6d1d066ede751034f69e2043b8207e515a3: Status 404 returned error can't find the container with id 90e5a7aa85a83f16b2c1c3e02f28d6d1d066ede751034f69e2043b8207e515a3
Aug 04 16:11:32.407419 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.407393    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"]
Aug 04 16:11:32.419900 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:11:32.419865    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19996ec8_4290_4e5a_8b24_3e5afcf20642.slice/crio-1f8078aea3789f4e758e779f925d1e9fa514823305569fcdcfde1d7e3079a2a9 WatchSource:0}: Error finding container 1f8078aea3789f4e758e779f925d1e9fa514823305569fcdcfde1d7e3079a2a9: Status 404 returned error can't find the container with id 1f8078aea3789f4e758e779f925d1e9fa514823305569fcdcfde1d7e3079a2a9
Aug 04 16:11:32.434482 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.434436    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl" event={"ID":"19996ec8-4290-4e5a-8b24-3e5afcf20642","Type":"ContainerStarted","Data":"1f8078aea3789f4e758e779f925d1e9fa514823305569fcdcfde1d7e3079a2a9"}
Aug 04 16:11:32.435786 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:32.435757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps" event={"ID":"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5","Type":"ContainerStarted","Data":"90e5a7aa85a83f16b2c1c3e02f28d6d1d066ede751034f69e2043b8207e515a3"}
Aug 04 16:11:33.444344 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:33.444306    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl" event={"ID":"19996ec8-4290-4e5a-8b24-3e5afcf20642","Type":"ContainerStarted","Data":"a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069"}
Aug 04 16:11:33.444870 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:33.444483    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:33.446062 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:33.446034    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps" event={"ID":"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5","Type":"ContainerStarted","Data":"acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce"}
Aug 04 16:11:33.446181 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:33.446097    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:33.463211 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:33.463149    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl" podStartSLOduration=2.463132881 podStartE2EDuration="2.463132881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:11:33.461048869 +0000 UTC m=+3635.967346938" watchObservedRunningTime="2026-08-04 16:11:33.463132881 +0000 UTC m=+3635.969430971"
Aug 04 16:11:33.479079 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:33.479021    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps" podStartSLOduration=2.479001746 podStartE2EDuration="2.479001746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:11:33.477778824 +0000 UTC m=+3635.984076881" watchObservedRunningTime="2026-08-04 16:11:33.479001746 +0000 UTC m=+3635.985299802"
Aug 04 16:11:34.922603 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:34.922567    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:11:44.453793 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:44.453756    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:11:44.454286 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:44.453832    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:11:55.763785 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:55.763750    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"]
Aug 04 16:11:55.767374 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:55.767352    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:55.769902 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:55.769874    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 16:11:55.770024 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:55.769877    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 16:11:55.776223 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:55.776191    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"]
Aug 04 16:11:55.933851 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:55.933785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488084b-8c77-4c47-90d7-4e7a16414b36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr\" (UID: \"4488084b-8c77-4c47-90d7-4e7a16414b36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:56.034829 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:56.034736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488084b-8c77-4c47-90d7-4e7a16414b36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr\" (UID: \"4488084b-8c77-4c47-90d7-4e7a16414b36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:56.035414 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:56.035394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488084b-8c77-4c47-90d7-4e7a16414b36-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr\" (UID: \"4488084b-8c77-4c47-90d7-4e7a16414b36\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:56.080721 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:56.080682    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:56.431031 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:56.431003    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"]
Aug 04 16:11:56.433223 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:11:56.433191    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4488084b_8c77_4c47_90d7_4e7a16414b36.slice/crio-b90df9b5f178e7e692d2184311df522b41cc64158a836611e204f965e77a44c0 WatchSource:0}: Error finding container b90df9b5f178e7e692d2184311df522b41cc64158a836611e204f965e77a44c0: Status 404 returned error can't find the container with id b90df9b5f178e7e692d2184311df522b41cc64158a836611e204f965e77a44c0
Aug 04 16:11:56.560081 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:56.560045    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr" event={"ID":"4488084b-8c77-4c47-90d7-4e7a16414b36","Type":"ContainerStarted","Data":"b90df9b5f178e7e692d2184311df522b41cc64158a836611e204f965e77a44c0"}
Aug 04 16:11:57.567521 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:57.567427    2569 generic.go:356] "Generic (PLEG): container finished" podID="4488084b-8c77-4c47-90d7-4e7a16414b36" containerID="e53d9fff240de9159220852160730a54ef8e3794328700688eb0753568b928ad" exitCode=0
Aug 04 16:11:57.567938 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:57.567519    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr" event={"ID":"4488084b-8c77-4c47-90d7-4e7a16414b36","Type":"ContainerDied","Data":"e53d9fff240de9159220852160730a54ef8e3794328700688eb0753568b928ad"}
Aug 04 16:11:58.712535 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:58.712510    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:58.862952 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:58.862859    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488084b-8c77-4c47-90d7-4e7a16414b36-maas-api-service-ca\") pod \"4488084b-8c77-4c47-90d7-4e7a16414b36\" (UID: \"4488084b-8c77-4c47-90d7-4e7a16414b36\") "
Aug 04 16:11:58.863290 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:58.863265    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4488084b-8c77-4c47-90d7-4e7a16414b36-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4488084b-8c77-4c47-90d7-4e7a16414b36" (UID: "4488084b-8c77-4c47-90d7-4e7a16414b36"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:11:58.963866 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:58.963827    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488084b-8c77-4c47-90d7-4e7a16414b36-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:11:59.579096 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:59.579058    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr" event={"ID":"4488084b-8c77-4c47-90d7-4e7a16414b36","Type":"ContainerDied","Data":"b90df9b5f178e7e692d2184311df522b41cc64158a836611e204f965e77a44c0"}
Aug 04 16:11:59.579096 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:59.579084    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"
Aug 04 16:11:59.579319 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:11:59.579117    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b90df9b5f178e7e692d2184311df522b41cc64158a836611e204f965e77a44c0"
Aug 04 16:12:01.893600 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:01.893566    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"]
Aug 04 16:12:01.894139 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:01.893823    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl" podUID="19996ec8-4290-4e5a-8b24-3e5afcf20642" containerName="payload-processing" containerID="cri-o://a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069" gracePeriod=30
Aug 04 16:12:01.903164 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:01.903128    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"]
Aug 04 16:12:01.903684 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:01.903599    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps" podUID="8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" containerName="payload-pre-processing" containerID="cri-o://acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce" gracePeriod=30
Aug 04 16:12:02.048020 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.047947    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"]
Aug 04 16:12:02.051332 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.051293    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8528c9-zp8hr"]
Aug 04 16:12:02.176043 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.176009    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:12:02.205987 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.205957    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:12:02.298296 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.298240    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19996ec8-4290-4e5a-8b24-3e5afcf20642-plugins-config-volume\") pod \"19996ec8-4290-4e5a-8b24-3e5afcf20642\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") "
Aug 04 16:12:02.298296 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.298311    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-df7xl\" (UniqueName: \"kubernetes.io/projected/19996ec8-4290-4e5a-8b24-3e5afcf20642-kube-api-access-df7xl\") pod \"19996ec8-4290-4e5a-8b24-3e5afcf20642\" (UID: \"19996ec8-4290-4e5a-8b24-3e5afcf20642\") "
Aug 04 16:12:02.298774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.298747    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19996ec8-4290-4e5a-8b24-3e5afcf20642-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19996ec8-4290-4e5a-8b24-3e5afcf20642" (UID: "19996ec8-4290-4e5a-8b24-3e5afcf20642"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:12:02.300435 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.300411    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19996ec8-4290-4e5a-8b24-3e5afcf20642-kube-api-access-df7xl" (OuterVolumeSpecName: "kube-api-access-df7xl") pod "19996ec8-4290-4e5a-8b24-3e5afcf20642" (UID: "19996ec8-4290-4e5a-8b24-3e5afcf20642"). InnerVolumeSpecName "kube-api-access-df7xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:12:02.399006 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.398895    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-plugins-config-volume\") pod \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") "
Aug 04 16:12:02.399006 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.399014    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72wjl\" (UniqueName: \"kubernetes.io/projected/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-kube-api-access-72wjl\") pod \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\" (UID: \"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5\") "
Aug 04 16:12:02.399265 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.399202    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19996ec8-4290-4e5a-8b24-3e5afcf20642-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:02.399265 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.399213    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df7xl\" (UniqueName: \"kubernetes.io/projected/19996ec8-4290-4e5a-8b24-3e5afcf20642-kube-api-access-df7xl\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:02.399407 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.399375    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" (UID: "8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:12:02.401103 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.401079    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-kube-api-access-72wjl" (OuterVolumeSpecName: "kube-api-access-72wjl") pod "8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" (UID: "8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5"). InnerVolumeSpecName "kube-api-access-72wjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:12:02.499769 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.499717    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:02.499769 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.499763    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72wjl\" (UniqueName: \"kubernetes.io/projected/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5-kube-api-access-72wjl\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:02.600014 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.599975    2569 generic.go:356] "Generic (PLEG): container finished" podID="19996ec8-4290-4e5a-8b24-3e5afcf20642" containerID="a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069" exitCode=0
Aug 04 16:12:02.600229 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.600035    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"
Aug 04 16:12:02.600229 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.600071    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl" event={"ID":"19996ec8-4290-4e5a-8b24-3e5afcf20642","Type":"ContainerDied","Data":"a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069"}
Aug 04 16:12:02.600229 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.600134    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl" event={"ID":"19996ec8-4290-4e5a-8b24-3e5afcf20642","Type":"ContainerDied","Data":"1f8078aea3789f4e758e779f925d1e9fa514823305569fcdcfde1d7e3079a2a9"}
Aug 04 16:12:02.600229 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.600159    2569 scope.go:117] "RemoveContainer" containerID="a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069"
Aug 04 16:12:02.601428 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.601384    2569 generic.go:356] "Generic (PLEG): container finished" podID="8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" containerID="acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce" exitCode=0
Aug 04 16:12:02.601428 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.601425    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps" event={"ID":"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5","Type":"ContainerDied","Data":"acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce"}
Aug 04 16:12:02.601574 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.601431    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"
Aug 04 16:12:02.601574 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.601465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps" event={"ID":"8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5","Type":"ContainerDied","Data":"90e5a7aa85a83f16b2c1c3e02f28d6d1d066ede751034f69e2043b8207e515a3"}
Aug 04 16:12:02.610926 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.610902    2569 scope.go:117] "RemoveContainer" containerID="a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069"
Aug 04 16:12:02.611346 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:12:02.611312    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069\": container with ID starting with a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069 not found: ID does not exist" containerID="a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069"
Aug 04 16:12:02.611455 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.611360    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069"} err="failed to get container status \"a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069\": rpc error: code = NotFound desc = could not find container \"a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069\": container with ID starting with a3b035be0b2dbdf6397b4bd77ef7b5016616ed7a4bbc077ec059f851da169069 not found: ID does not exist"
Aug 04 16:12:02.611455 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.611387    2569 scope.go:117] "RemoveContainer" containerID="acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce"
Aug 04 16:12:02.622605 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.622479    2569 scope.go:117] "RemoveContainer" containerID="acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce"
Aug 04 16:12:02.622973 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:12:02.622952    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce\": container with ID starting with acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce not found: ID does not exist" containerID="acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce"
Aug 04 16:12:02.623043 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.622987    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce"} err="failed to get container status \"acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce\": rpc error: code = NotFound desc = could not find container \"acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce\": container with ID starting with acbc4423afcde2ce5b75f5564bc3ba081bddf877e4c072519bc5978981a208ce not found: ID does not exist"
Aug 04 16:12:02.625126 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.625080    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"]
Aug 04 16:12:02.627749 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.627724    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8528c9-95fc4d8c4-5jldl"]
Aug 04 16:12:02.639087 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.639059    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"]
Aug 04 16:12:02.642763 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:02.642737    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8528c9-b96d94784-b4pps"]
Aug 04 16:12:03.964995 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:03.964961    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19996ec8-4290-4e5a-8b24-3e5afcf20642" path="/var/lib/kubelet/pods/19996ec8-4290-4e5a-8b24-3e5afcf20642/volumes"
Aug 04 16:12:03.965407 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:03.965308    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4488084b-8c77-4c47-90d7-4e7a16414b36" path="/var/lib/kubelet/pods/4488084b-8c77-4c47-90d7-4e7a16414b36/volumes"
Aug 04 16:12:03.965631 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:03.965616    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" path="/var/lib/kubelet/pods/8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5/volumes"
Aug 04 16:12:04.917695 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:04.917663    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:12:12.300091 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300046    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"]
Aug 04 16:12:12.300672 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300615    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" containerName="payload-pre-processing"
Aug 04 16:12:12.300672 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300660    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" containerName="payload-pre-processing"
Aug 04 16:12:12.300825 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300678    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4488084b-8c77-4c47-90d7-4e7a16414b36" containerName="revoke-keys"
Aug 04 16:12:12.300825 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300687    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4488084b-8c77-4c47-90d7-4e7a16414b36" containerName="revoke-keys"
Aug 04 16:12:12.300825 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300720    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19996ec8-4290-4e5a-8b24-3e5afcf20642" containerName="payload-processing"
Aug 04 16:12:12.300825 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300729    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="19996ec8-4290-4e5a-8b24-3e5afcf20642" containerName="payload-processing"
Aug 04 16:12:12.301028 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300847    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4488084b-8c77-4c47-90d7-4e7a16414b36" containerName="revoke-keys"
Aug 04 16:12:12.301028 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300861    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="19996ec8-4290-4e5a-8b24-3e5afcf20642" containerName="payload-processing"
Aug 04 16:12:12.301028 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.300873    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bbb7015-bcc6-43fd-8e85-5e80edc6f8e5" containerName="payload-pre-processing"
Aug 04 16:12:12.306021 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.305995    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:12.308666 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.308622    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 16:12:12.308834 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.308815    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 16:12:12.312493 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.312466    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"]
Aug 04 16:12:12.385508 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.385469    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9c00be-9c71-42ca-98ff-4848d2f86a15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5\" (UID: \"8b9c00be-9c71-42ca-98ff-4848d2f86a15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:12.486098 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.486053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9c00be-9c71-42ca-98ff-4848d2f86a15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5\" (UID: \"8b9c00be-9c71-42ca-98ff-4848d2f86a15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:12.486751 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.486732    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9c00be-9c71-42ca-98ff-4848d2f86a15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5\" (UID: \"8b9c00be-9c71-42ca-98ff-4848d2f86a15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:12.618004 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.617969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:12.764837 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:12.764582    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"]
Aug 04 16:12:12.767772 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:12:12.767733    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9c00be_9c71_42ca_98ff_4848d2f86a15.slice/crio-10663bc7e9973b6f67fab88cc8a780c13a6cd91b4998a4cc790485c2133ce67e WatchSource:0}: Error finding container 10663bc7e9973b6f67fab88cc8a780c13a6cd91b4998a4cc790485c2133ce67e: Status 404 returned error can't find the container with id 10663bc7e9973b6f67fab88cc8a780c13a6cd91b4998a4cc790485c2133ce67e
Aug 04 16:12:13.664228 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:13.664187    2569 generic.go:356] "Generic (PLEG): container finished" podID="8b9c00be-9c71-42ca-98ff-4848d2f86a15" containerID="667ed9b15c34e68e09e0953ac5fd743e22dd3fa67f517c4cb6cefdf0ffa31f00" exitCode=0
Aug 04 16:12:13.664629 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:13.664268    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5" event={"ID":"8b9c00be-9c71-42ca-98ff-4848d2f86a15","Type":"ContainerDied","Data":"667ed9b15c34e68e09e0953ac5fd743e22dd3fa67f517c4cb6cefdf0ffa31f00"}
Aug 04 16:12:13.664629 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:13.664324    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5" event={"ID":"8b9c00be-9c71-42ca-98ff-4848d2f86a15","Type":"ContainerStarted","Data":"10663bc7e9973b6f67fab88cc8a780c13a6cd91b4998a4cc790485c2133ce67e"}
Aug 04 16:12:14.804190 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:14.804160    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:14.910873 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:14.910835    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9c00be-9c71-42ca-98ff-4848d2f86a15-maas-api-service-ca\") pod \"8b9c00be-9c71-42ca-98ff-4848d2f86a15\" (UID: \"8b9c00be-9c71-42ca-98ff-4848d2f86a15\") "
Aug 04 16:12:14.911249 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:14.911221    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b9c00be-9c71-42ca-98ff-4848d2f86a15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b9c00be-9c71-42ca-98ff-4848d2f86a15" (UID: "8b9c00be-9c71-42ca-98ff-4848d2f86a15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:12:15.011767 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:15.011633    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b9c00be-9c71-42ca-98ff-4848d2f86a15-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:15.675330 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:15.675298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5" event={"ID":"8b9c00be-9c71-42ca-98ff-4848d2f86a15","Type":"ContainerDied","Data":"10663bc7e9973b6f67fab88cc8a780c13a6cd91b4998a4cc790485c2133ce67e"}
Aug 04 16:12:15.675531 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:15.675349    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"
Aug 04 16:12:15.675531 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:15.675353    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10663bc7e9973b6f67fab88cc8a780c13a6cd91b4998a4cc790485c2133ce67e"
Aug 04 16:12:25.920183 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:25.920081    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:12:27.886990 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:27.886956    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"]
Aug 04 16:12:27.887509 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:27.887286    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4" podUID="ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" containerName="payload-processing" containerID="cri-o://8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530" gracePeriod=30
Aug 04 16:12:27.893775 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:27.893198    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"]
Aug 04 16:12:27.893775 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:27.893455    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn" podUID="d80c4390-2b1a-4daa-8735-7ce421a9d7c9" containerName="payload-pre-processing" containerID="cri-o://b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4" gracePeriod=30
Aug 04 16:12:28.026320 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.026243    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"]
Aug 04 16:12:28.029446 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.029412    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8ff9cf-jmdx5"]
Aug 04 16:12:28.164104 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.164074    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:12:28.188123 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.188100    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:12:28.246573 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.246538    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5bwk\" (UniqueName: \"kubernetes.io/projected/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-kube-api-access-t5bwk\") pod \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") "
Aug 04 16:12:28.246766 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.246674    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvl2v\" (UniqueName: \"kubernetes.io/projected/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-kube-api-access-mvl2v\") pod \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") "
Aug 04 16:12:28.246766 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.246722    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-plugins-config-volume\") pod \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\" (UID: \"d80c4390-2b1a-4daa-8735-7ce421a9d7c9\") "
Aug 04 16:12:28.246766 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.246748    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-plugins-config-volume\") pod \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\" (UID: \"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a\") "
Aug 04 16:12:28.247173 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.247143    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d80c4390-2b1a-4daa-8735-7ce421a9d7c9" (UID: "d80c4390-2b1a-4daa-8735-7ce421a9d7c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:12:28.247310 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.247237    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" (UID: "ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:12:28.248920 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.248895    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-kube-api-access-t5bwk" (OuterVolumeSpecName: "kube-api-access-t5bwk") pod "ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" (UID: "ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a"). InnerVolumeSpecName "kube-api-access-t5bwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:12:28.249016 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.248934    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-kube-api-access-mvl2v" (OuterVolumeSpecName: "kube-api-access-mvl2v") pod "d80c4390-2b1a-4daa-8735-7ce421a9d7c9" (UID: "d80c4390-2b1a-4daa-8735-7ce421a9d7c9"). InnerVolumeSpecName "kube-api-access-mvl2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:12:28.348250 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.348212    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvl2v\" (UniqueName: \"kubernetes.io/projected/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-kube-api-access-mvl2v\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:28.348250 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.348245    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d80c4390-2b1a-4daa-8735-7ce421a9d7c9-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:28.348250 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.348256    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:28.348509 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.348265    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5bwk\" (UniqueName: \"kubernetes.io/projected/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a-kube-api-access-t5bwk\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:28.747927 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.747887    2569 generic.go:356] "Generic (PLEG): container finished" podID="d80c4390-2b1a-4daa-8735-7ce421a9d7c9" containerID="b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4" exitCode=0
Aug 04 16:12:28.748099 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.747931    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"
Aug 04 16:12:28.748099 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.747976    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn" event={"ID":"d80c4390-2b1a-4daa-8735-7ce421a9d7c9","Type":"ContainerDied","Data":"b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4"}
Aug 04 16:12:28.748099 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.748030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn" event={"ID":"d80c4390-2b1a-4daa-8735-7ce421a9d7c9","Type":"ContainerDied","Data":"5d12bbdc4ea49caa2a51d5b56b2ccfb5c08ce5c85c7a93f731e4e55dc0ec40f4"}
Aug 04 16:12:28.748099 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.748063    2569 scope.go:117] "RemoveContainer" containerID="b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4"
Aug 04 16:12:28.749438 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.749414    2569 generic.go:356] "Generic (PLEG): container finished" podID="ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" containerID="8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530" exitCode=0
Aug 04 16:12:28.749522 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.749466    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"
Aug 04 16:12:28.749575 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.749461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4" event={"ID":"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a","Type":"ContainerDied","Data":"8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530"}
Aug 04 16:12:28.749615 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.749592    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4" event={"ID":"ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a","Type":"ContainerDied","Data":"fd06db053141f1f76dca07bb63aa33e2774c90a6be63ff9cafbbbd845e72c595"}
Aug 04 16:12:28.762176 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.762149    2569 scope.go:117] "RemoveContainer" containerID="b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4"
Aug 04 16:12:28.763008 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:12:28.762985    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4\": container with ID starting with b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4 not found: ID does not exist" containerID="b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4"
Aug 04 16:12:28.763103 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.763017    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4"} err="failed to get container status \"b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4\": rpc error: code = NotFound desc = could not find container \"b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4\": container with ID starting with b2e48296a9f18a02e21c001cd9412d402005dd8f80c6fc3867fa4c77bc251cc4 not found: ID does not exist"
Aug 04 16:12:28.763103 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.763035    2569 scope.go:117] "RemoveContainer" containerID="8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530"
Aug 04 16:12:28.772081 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.772060    2569 scope.go:117] "RemoveContainer" containerID="8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530"
Aug 04 16:12:28.772400 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:12:28.772381    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530\": container with ID starting with 8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530 not found: ID does not exist" containerID="8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530"
Aug 04 16:12:28.772474 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.772410    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530"} err="failed to get container status \"8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530\": rpc error: code = NotFound desc = could not find container \"8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530\": container with ID starting with 8d387047becd89111e485fd13e5d29bbf0d303ed94be8f8c7bb40b59ca21d530 not found: ID does not exist"
Aug 04 16:12:28.772915 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.772893    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"]
Aug 04 16:12:28.776502 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.776474    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8ff9cf-799685b948-kg2hn"]
Aug 04 16:12:28.787773 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.787735    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"]
Aug 04 16:12:28.793780 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:28.793750    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8ff9cf-849c8cf58b-t8zq4"]
Aug 04 16:12:29.964751 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:29.964718    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b9c00be-9c71-42ca-98ff-4848d2f86a15" path="/var/lib/kubelet/pods/8b9c00be-9c71-42ca-98ff-4848d2f86a15/volumes"
Aug 04 16:12:29.965233 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:29.965061    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" path="/var/lib/kubelet/pods/ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a/volumes"
Aug 04 16:12:29.965383 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:29.965370    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d80c4390-2b1a-4daa-8735-7ce421a9d7c9" path="/var/lib/kubelet/pods/d80c4390-2b1a-4daa-8735-7ce421a9d7c9/volumes"
Aug 04 16:12:33.405657 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:33.405611    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"]
Aug 04 16:12:34.726337 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:34.726298    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:12:48.425573 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.425518    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2" podUID="6587d644-e186-4878-bd6c-38f37c99e5bc" containerName="main" containerID="cri-o://04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52" gracePeriod=45
Aug 04 16:12:48.686352 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.686289    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:12:48.742079 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742044    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qqxw\" (UniqueName: \"kubernetes.io/projected/6587d644-e186-4878-bd6c-38f37c99e5bc-kube-api-access-8qqxw\") pod \"6587d644-e186-4878-bd6c-38f37c99e5bc\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") "
Aug 04 16:12:48.742302 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742117    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-dshm\") pod \"6587d644-e186-4878-bd6c-38f37c99e5bc\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") "
Aug 04 16:12:48.742302 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742154    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6587d644-e186-4878-bd6c-38f37c99e5bc-tls-certs\") pod \"6587d644-e186-4878-bd6c-38f37c99e5bc\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") "
Aug 04 16:12:48.742302 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742176    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-tmp-dir\") pod \"6587d644-e186-4878-bd6c-38f37c99e5bc\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") "
Aug 04 16:12:48.742302 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742250    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-model-cache\") pod \"6587d644-e186-4878-bd6c-38f37c99e5bc\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") "
Aug 04 16:12:48.742302 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742279    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-home\") pod \"6587d644-e186-4878-bd6c-38f37c99e5bc\" (UID: \"6587d644-e186-4878-bd6c-38f37c99e5bc\") "
Aug 04 16:12:48.742885 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742515    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6587d644-e186-4878-bd6c-38f37c99e5bc" (UID: "6587d644-e186-4878-bd6c-38f37c99e5bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:12:48.742885 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742539    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "6587d644-e186-4878-bd6c-38f37c99e5bc" (UID: "6587d644-e186-4878-bd6c-38f37c99e5bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:12:48.742885 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.742756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-home" (OuterVolumeSpecName: "home") pod "6587d644-e186-4878-bd6c-38f37c99e5bc" (UID: "6587d644-e186-4878-bd6c-38f37c99e5bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:12:48.744451 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.744427    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-dshm" (OuterVolumeSpecName: "dshm") pod "6587d644-e186-4878-bd6c-38f37c99e5bc" (UID: "6587d644-e186-4878-bd6c-38f37c99e5bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 04 16:12:48.744556 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.744446    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6587d644-e186-4878-bd6c-38f37c99e5bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6587d644-e186-4878-bd6c-38f37c99e5bc" (UID: "6587d644-e186-4878-bd6c-38f37c99e5bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 04 16:12:48.744556 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.744538    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6587d644-e186-4878-bd6c-38f37c99e5bc-kube-api-access-8qqxw" (OuterVolumeSpecName: "kube-api-access-8qqxw") pod "6587d644-e186-4878-bd6c-38f37c99e5bc" (UID: "6587d644-e186-4878-bd6c-38f37c99e5bc"). InnerVolumeSpecName "kube-api-access-8qqxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:12:48.843298 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.843257    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-model-cache\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:48.843298 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.843289    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-home\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:48.843298 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.843299    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qqxw\" (UniqueName: \"kubernetes.io/projected/6587d644-e186-4878-bd6c-38f37c99e5bc-kube-api-access-8qqxw\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:48.843298 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.843309    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-dshm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:48.843588 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.843318    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6587d644-e186-4878-bd6c-38f37c99e5bc-tls-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:48.843588 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.843328    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6587d644-e186-4878-bd6c-38f37c99e5bc-tmp-dir\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:12:48.852194 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.852154    2569 generic.go:356] "Generic (PLEG): container finished" podID="6587d644-e186-4878-bd6c-38f37c99e5bc" containerID="04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52" exitCode=0
Aug 04 16:12:48.852384 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.852202    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"
Aug 04 16:12:48.852384 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.852235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2" event={"ID":"6587d644-e186-4878-bd6c-38f37c99e5bc","Type":"ContainerDied","Data":"04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52"}
Aug 04 16:12:48.852384 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.852278    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2" event={"ID":"6587d644-e186-4878-bd6c-38f37c99e5bc","Type":"ContainerDied","Data":"710c3954dba1aeecdb42373edd4c3a4ede171e2a5ebc34117640eac3c1d31c7e"}
Aug 04 16:12:48.852384 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.852300    2569 scope.go:117] "RemoveContainer" containerID="04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52"
Aug 04 16:12:48.862106 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.862085    2569 scope.go:117] "RemoveContainer" containerID="04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52"
Aug 04 16:12:48.862390 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:12:48.862373    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52\": container with ID starting with 04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52 not found: ID does not exist" containerID="04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52"
Aug 04 16:12:48.862449 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.862399    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52"} err="failed to get container status \"04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52\": rpc error: code = NotFound desc = could not find container \"04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52\": container with ID starting with 04d094ed9f1090516a77d669c9d0cec3ed935c7a9e5a593ffc3cedc0de66bc52 not found: ID does not exist"
Aug 04 16:12:48.875329 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.875300    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"]
Aug 04 16:12:48.878038 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:48.878013    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-8ff9cf/ipp-route-8ff9cf-kserve-766bc69747-t7br2"]
Aug 04 16:12:49.964991 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:12:49.964959    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6587d644-e186-4878-bd6c-38f37c99e5bc" path="/var/lib/kubelet/pods/6587d644-e186-4878-bd6c-38f37c99e5bc/volumes"
Aug 04 16:13:17.802833 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.802793    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"]
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803231    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b9c00be-9c71-42ca-98ff-4848d2f86a15" containerName="revoke-keys"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803242    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b9c00be-9c71-42ca-98ff-4848d2f86a15" containerName="revoke-keys"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803253    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6587d644-e186-4878-bd6c-38f37c99e5bc" containerName="main"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803259    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6587d644-e186-4878-bd6c-38f37c99e5bc" containerName="main"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803279    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80c4390-2b1a-4daa-8735-7ce421a9d7c9" containerName="payload-pre-processing"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803285    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80c4390-2b1a-4daa-8735-7ce421a9d7c9" containerName="payload-pre-processing"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803293    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" containerName="payload-processing"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803299    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" containerName="payload-processing"
Aug 04 16:13:17.803348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803354    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d80c4390-2b1a-4daa-8735-7ce421a9d7c9" containerName="payload-pre-processing"
Aug 04 16:13:17.803716 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803362    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b9c00be-9c71-42ca-98ff-4848d2f86a15" containerName="revoke-keys"
Aug 04 16:13:17.803716 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803369    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce527452-4ed5-4cfd-b1a1-e289ae2b6b9a" containerName="payload-processing"
Aug 04 16:13:17.803716 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.803378    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6587d644-e186-4878-bd6c-38f37c99e5bc" containerName="main"
Aug 04 16:13:17.806473 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.806455    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:17.809068 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.809044    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 16:13:17.809228 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.809173    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 16:13:17.817148 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.817115    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"]
Aug 04 16:13:17.932999 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:17.932951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d6f3b4b-1d3a-4240-8924-6e501d1607c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68\" (UID: \"8d6f3b4b-1d3a-4240-8924-6e501d1607c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:18.033888 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:18.033845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d6f3b4b-1d3a-4240-8924-6e501d1607c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68\" (UID: \"8d6f3b4b-1d3a-4240-8924-6e501d1607c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:18.034535 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:18.034508    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d6f3b4b-1d3a-4240-8924-6e501d1607c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68\" (UID: \"8d6f3b4b-1d3a-4240-8924-6e501d1607c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:18.117788 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:18.117757    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:18.260766 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:18.260737    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"]
Aug 04 16:13:18.263799 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:13:18.263766    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d6f3b4b_1d3a_4240_8924_6e501d1607c6.slice/crio-fb6f0a3acb9a934f2103723558fa7066621b81b891274aa7827dfbed43b8fe2f WatchSource:0}: Error finding container fb6f0a3acb9a934f2103723558fa7066621b81b891274aa7827dfbed43b8fe2f: Status 404 returned error can't find the container with id fb6f0a3acb9a934f2103723558fa7066621b81b891274aa7827dfbed43b8fe2f
Aug 04 16:13:19.003426 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:19.003376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68" event={"ID":"8d6f3b4b-1d3a-4240-8924-6e501d1607c6","Type":"ContainerStarted","Data":"845933bfda8475fd1011bab8a9c5be7e96572577317148a2fb618ebea8731933"}
Aug 04 16:13:19.003850 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:19.003447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68" event={"ID":"8d6f3b4b-1d3a-4240-8924-6e501d1607c6","Type":"ContainerStarted","Data":"fb6f0a3acb9a934f2103723558fa7066621b81b891274aa7827dfbed43b8fe2f"}
Aug 04 16:13:20.010391 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:20.010350    2569 generic.go:356] "Generic (PLEG): container finished" podID="8d6f3b4b-1d3a-4240-8924-6e501d1607c6" containerID="845933bfda8475fd1011bab8a9c5be7e96572577317148a2fb618ebea8731933" exitCode=0
Aug 04 16:13:20.010819 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:20.010438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68" event={"ID":"8d6f3b4b-1d3a-4240-8924-6e501d1607c6","Type":"ContainerDied","Data":"845933bfda8475fd1011bab8a9c5be7e96572577317148a2fb618ebea8731933"}
Aug 04 16:13:21.159668 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:21.159627    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:21.162715 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:21.162694    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d6f3b4b-1d3a-4240-8924-6e501d1607c6-maas-api-service-ca\") pod \"8d6f3b4b-1d3a-4240-8924-6e501d1607c6\" (UID: \"8d6f3b4b-1d3a-4240-8924-6e501d1607c6\") "
Aug 04 16:13:21.163042 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:21.163020    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d6f3b4b-1d3a-4240-8924-6e501d1607c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d6f3b4b-1d3a-4240-8924-6e501d1607c6" (UID: "8d6f3b4b-1d3a-4240-8924-6e501d1607c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:21.264192 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:21.264158    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d6f3b4b-1d3a-4240-8924-6e501d1607c6-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:22.020948 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:22.020909    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68" event={"ID":"8d6f3b4b-1d3a-4240-8924-6e501d1607c6","Type":"ContainerDied","Data":"fb6f0a3acb9a934f2103723558fa7066621b81b891274aa7827dfbed43b8fe2f"}
Aug 04 16:13:22.020948 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:22.020936    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"
Aug 04 16:13:22.021163 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:22.020969    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb6f0a3acb9a934f2103723558fa7066621b81b891274aa7827dfbed43b8fe2f"
Aug 04 16:13:28.462606 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.462563    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"]
Aug 04 16:13:28.463271 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.462902    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5" podUID="02fa956f-4692-44bf-ac25-ee1ddb3cfde8" containerName="payload-processing" containerID="cri-o://f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c" gracePeriod=30
Aug 04 16:13:28.474785 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.474749    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"]
Aug 04 16:13:28.475528 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.475022    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs" podUID="56ff87fc-bba4-4f53-9998-04329208c002" containerName="payload-pre-processing" containerID="cri-o://ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044" gracePeriod=30
Aug 04 16:13:28.711718 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.711678    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"]
Aug 04 16:13:28.713383 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.713309    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2b50f0-bdm68"]
Aug 04 16:13:28.769769 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.769739    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:13:28.772794 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.772769    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:13:28.835690 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.835632    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbqr6\" (UniqueName: \"kubernetes.io/projected/56ff87fc-bba4-4f53-9998-04329208c002-kube-api-access-rbqr6\") pod \"56ff87fc-bba4-4f53-9998-04329208c002\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") "
Aug 04 16:13:28.835894 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.835719    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vxsm\" (UniqueName: \"kubernetes.io/projected/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-kube-api-access-8vxsm\") pod \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") "
Aug 04 16:13:28.835894 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.835794    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ff87fc-bba4-4f53-9998-04329208c002-plugins-config-volume\") pod \"56ff87fc-bba4-4f53-9998-04329208c002\" (UID: \"56ff87fc-bba4-4f53-9998-04329208c002\") "
Aug 04 16:13:28.835894 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.835879    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-plugins-config-volume\") pod \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\" (UID: \"02fa956f-4692-44bf-ac25-ee1ddb3cfde8\") "
Aug 04 16:13:28.836256 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.836226    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02fa956f-4692-44bf-ac25-ee1ddb3cfde8" (UID: "02fa956f-4692-44bf-ac25-ee1ddb3cfde8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:28.836380 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.836226    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ff87fc-bba4-4f53-9998-04329208c002-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56ff87fc-bba4-4f53-9998-04329208c002" (UID: "56ff87fc-bba4-4f53-9998-04329208c002"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:28.837933 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.837910    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-kube-api-access-8vxsm" (OuterVolumeSpecName: "kube-api-access-8vxsm") pod "02fa956f-4692-44bf-ac25-ee1ddb3cfde8" (UID: "02fa956f-4692-44bf-ac25-ee1ddb3cfde8"). InnerVolumeSpecName "kube-api-access-8vxsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:13:28.838053 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.837922    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ff87fc-bba4-4f53-9998-04329208c002-kube-api-access-rbqr6" (OuterVolumeSpecName: "kube-api-access-rbqr6") pod "56ff87fc-bba4-4f53-9998-04329208c002" (UID: "56ff87fc-bba4-4f53-9998-04329208c002"). InnerVolumeSpecName "kube-api-access-rbqr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:13:28.937134 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.937094    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vxsm\" (UniqueName: \"kubernetes.io/projected/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-kube-api-access-8vxsm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:28.937134 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.937127    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ff87fc-bba4-4f53-9998-04329208c002-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:28.937134 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.937140    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02fa956f-4692-44bf-ac25-ee1ddb3cfde8-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:28.937384 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:28.937149    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbqr6\" (UniqueName: \"kubernetes.io/projected/56ff87fc-bba4-4f53-9998-04329208c002-kube-api-access-rbqr6\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:29.058139 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.058038    2569 generic.go:356] "Generic (PLEG): container finished" podID="02fa956f-4692-44bf-ac25-ee1ddb3cfde8" containerID="f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c" exitCode=0
Aug 04 16:13:29.058139 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.058098    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"
Aug 04 16:13:29.058381 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.058134    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5" event={"ID":"02fa956f-4692-44bf-ac25-ee1ddb3cfde8","Type":"ContainerDied","Data":"f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c"}
Aug 04 16:13:29.058381 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.058186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5" event={"ID":"02fa956f-4692-44bf-ac25-ee1ddb3cfde8","Type":"ContainerDied","Data":"1a0357028c555d6b5f797dee42e1d77fafc14c9521891fe3096b4d1fd5dc1f79"}
Aug 04 16:13:29.058381 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.058217    2569 scope.go:117] "RemoveContainer" containerID="f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c"
Aug 04 16:13:29.059602 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.059579    2569 generic.go:356] "Generic (PLEG): container finished" podID="56ff87fc-bba4-4f53-9998-04329208c002" containerID="ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044" exitCode=0
Aug 04 16:13:29.059752 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.059654    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"
Aug 04 16:13:29.059752 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.059704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs" event={"ID":"56ff87fc-bba4-4f53-9998-04329208c002","Type":"ContainerDied","Data":"ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044"}
Aug 04 16:13:29.059839 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.059776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs" event={"ID":"56ff87fc-bba4-4f53-9998-04329208c002","Type":"ContainerDied","Data":"17b1b38fdc197961e840d97bb7c879328740ff08f630349e4eed1cc098e4c114"}
Aug 04 16:13:29.071357 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.071331    2569 scope.go:117] "RemoveContainer" containerID="f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c"
Aug 04 16:13:29.071862 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:13:29.071832    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c\": container with ID starting with f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c not found: ID does not exist" containerID="f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c"
Aug 04 16:13:29.071946 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.071876    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c"} err="failed to get container status \"f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c\": rpc error: code = NotFound desc = could not find container \"f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c\": container with ID starting with f16c54fff3b14d4ec2e3335772efb5ca4b5a827f2b983d970ac08adad9b5ca2c not found: ID does not exist"
Aug 04 16:13:29.071946 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.071902    2569 scope.go:117] "RemoveContainer" containerID="ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044"
Aug 04 16:13:29.081368 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.081349    2569 scope.go:117] "RemoveContainer" containerID="ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044"
Aug 04 16:13:29.081690 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:13:29.081662    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044\": container with ID starting with ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044 not found: ID does not exist" containerID="ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044"
Aug 04 16:13:29.081769 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.081700    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044"} err="failed to get container status \"ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044\": rpc error: code = NotFound desc = could not find container \"ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044\": container with ID starting with ce14a55b671e8d4422106e894febdbbeaca96b44a1803aa26317f51b8aba6044 not found: ID does not exist"
Aug 04 16:13:29.104561 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.104526    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"]
Aug 04 16:13:29.107907 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.107881    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2b50f0-6b476cdc78-95mb5"]
Aug 04 16:13:29.127288 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.127244    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"]
Aug 04 16:13:29.129892 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.129868    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2b50f0-6d75d697fb-kc9xs"]
Aug 04 16:13:29.967267 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.967228    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02fa956f-4692-44bf-ac25-ee1ddb3cfde8" path="/var/lib/kubelet/pods/02fa956f-4692-44bf-ac25-ee1ddb3cfde8/volumes"
Aug 04 16:13:29.967694 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.967598    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56ff87fc-bba4-4f53-9998-04329208c002" path="/var/lib/kubelet/pods/56ff87fc-bba4-4f53-9998-04329208c002/volumes"
Aug 04 16:13:29.967952 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:29.967937    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d6f3b4b-1d3a-4240-8924-6e501d1607c6" path="/var/lib/kubelet/pods/8d6f3b4b-1d3a-4240-8924-6e501d1607c6/volumes"
Aug 04 16:13:31.153773 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:31.153733    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:13:35.851208 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851169    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"]
Aug 04 16:13:35.851628 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851560    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d6f3b4b-1d3a-4240-8924-6e501d1607c6" containerName="revoke-keys"
Aug 04 16:13:35.851628 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851570    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d6f3b4b-1d3a-4240-8924-6e501d1607c6" containerName="revoke-keys"
Aug 04 16:13:35.851628 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851579    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ff87fc-bba4-4f53-9998-04329208c002" containerName="payload-pre-processing"
Aug 04 16:13:35.851628 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851585    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ff87fc-bba4-4f53-9998-04329208c002" containerName="payload-pre-processing"
Aug 04 16:13:35.851628 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851599    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02fa956f-4692-44bf-ac25-ee1ddb3cfde8" containerName="payload-processing"
Aug 04 16:13:35.851628 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851604    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="02fa956f-4692-44bf-ac25-ee1ddb3cfde8" containerName="payload-processing"
Aug 04 16:13:35.851859 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851672    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="02fa956f-4692-44bf-ac25-ee1ddb3cfde8" containerName="payload-processing"
Aug 04 16:13:35.851859 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851682    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d6f3b4b-1d3a-4240-8924-6e501d1607c6" containerName="revoke-keys"
Aug 04 16:13:35.851859 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.851688    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ff87fc-bba4-4f53-9998-04329208c002" containerName="payload-pre-processing"
Aug 04 16:13:35.854684 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.854662    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:35.857819 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.857796    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 16:13:35.858125 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.858104    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 16:13:35.868315 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.868279    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"]
Aug 04 16:13:35.896155 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.896113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a23a29-fbee-4609-8c8c-dc0da38c5e40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw\" (UID: \"b5a23a29-fbee-4609-8c8c-dc0da38c5e40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:35.997619 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.997571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a23a29-fbee-4609-8c8c-dc0da38c5e40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw\" (UID: \"b5a23a29-fbee-4609-8c8c-dc0da38c5e40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:35.998261 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:35.998241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a23a29-fbee-4609-8c8c-dc0da38c5e40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw\" (UID: \"b5a23a29-fbee-4609-8c8c-dc0da38c5e40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:36.165650 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:36.165600    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:36.305747 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:36.305723    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"]
Aug 04 16:13:36.307841 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:13:36.307811    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a23a29_fbee_4609_8c8c_dc0da38c5e40.slice/crio-51393db141d1737ac12d3598f23421f3bc6a4ea1599bb9eb675a7069e6755ec0 WatchSource:0}: Error finding container 51393db141d1737ac12d3598f23421f3bc6a4ea1599bb9eb675a7069e6755ec0: Status 404 returned error can't find the container with id 51393db141d1737ac12d3598f23421f3bc6a4ea1599bb9eb675a7069e6755ec0
Aug 04 16:13:37.104312 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:37.104270    2569 generic.go:356] "Generic (PLEG): container finished" podID="b5a23a29-fbee-4609-8c8c-dc0da38c5e40" containerID="9e3ac4cff2fa0d6dd99cf410b5adabb778cae6e004f87457c479e91269cc7b1b" exitCode=0
Aug 04 16:13:37.104769 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:37.104332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw" event={"ID":"b5a23a29-fbee-4609-8c8c-dc0da38c5e40","Type":"ContainerDied","Data":"9e3ac4cff2fa0d6dd99cf410b5adabb778cae6e004f87457c479e91269cc7b1b"}
Aug 04 16:13:37.104769 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:37.104385    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw" event={"ID":"b5a23a29-fbee-4609-8c8c-dc0da38c5e40","Type":"ContainerStarted","Data":"51393db141d1737ac12d3598f23421f3bc6a4ea1599bb9eb675a7069e6755ec0"}
Aug 04 16:13:38.246981 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:38.246953    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:38.317242 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:38.317207    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a23a29-fbee-4609-8c8c-dc0da38c5e40-maas-api-service-ca\") pod \"b5a23a29-fbee-4609-8c8c-dc0da38c5e40\" (UID: \"b5a23a29-fbee-4609-8c8c-dc0da38c5e40\") "
Aug 04 16:13:38.317602 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:38.317577    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a23a29-fbee-4609-8c8c-dc0da38c5e40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b5a23a29-fbee-4609-8c8c-dc0da38c5e40" (UID: "b5a23a29-fbee-4609-8c8c-dc0da38c5e40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:38.418319 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:38.418276    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a23a29-fbee-4609-8c8c-dc0da38c5e40-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:39.118048 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:39.118007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw" event={"ID":"b5a23a29-fbee-4609-8c8c-dc0da38c5e40","Type":"ContainerDied","Data":"51393db141d1737ac12d3598f23421f3bc6a4ea1599bb9eb675a7069e6755ec0"}
Aug 04 16:13:39.118048 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:39.118034    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"
Aug 04 16:13:39.118315 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:39.118068    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51393db141d1737ac12d3598f23421f3bc6a4ea1599bb9eb675a7069e6755ec0"
Aug 04 16:13:46.556850 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.556809    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"]
Aug 04 16:13:46.557380 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.557041    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b" podUID="ead24e1a-66f3-460c-8f3c-cb25c6afe89a" containerName="payload-processing" containerID="cri-o://d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672" gracePeriod=30
Aug 04 16:13:46.565504 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.565477    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"]
Aug 04 16:13:46.565778 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.565736    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw" podUID="bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" containerName="payload-pre-processing" containerID="cri-o://31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232" gracePeriod=30
Aug 04 16:13:46.845613 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.845572    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 16:13:46.848743 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.848716    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"]
Aug 04 16:13:46.849167 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.849152    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 16:13:46.858302 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.858271    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67a15f-jmvcw"]
Aug 04 16:13:46.993760 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.993715    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crm22\" (UniqueName: \"kubernetes.io/projected/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-kube-api-access-crm22\") pod \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") "
Aug 04 16:13:46.993959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.993774    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-plugins-config-volume\") pod \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\" (UID: \"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72\") "
Aug 04 16:13:46.993959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.993811    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-plugins-config-volume\") pod \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") "
Aug 04 16:13:46.993959 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.993836    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcnxm\" (UniqueName: \"kubernetes.io/projected/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-kube-api-access-xcnxm\") pod \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\" (UID: \"ead24e1a-66f3-460c-8f3c-cb25c6afe89a\") "
Aug 04 16:13:46.994241 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.994211    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" (UID: "bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:46.994241 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.994225    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ead24e1a-66f3-460c-8f3c-cb25c6afe89a" (UID: "ead24e1a-66f3-460c-8f3c-cb25c6afe89a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:46.996080 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.996051    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-kube-api-access-xcnxm" (OuterVolumeSpecName: "kube-api-access-xcnxm") pod "ead24e1a-66f3-460c-8f3c-cb25c6afe89a" (UID: "ead24e1a-66f3-460c-8f3c-cb25c6afe89a"). InnerVolumeSpecName "kube-api-access-xcnxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:13:46.996080 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:46.996069    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-kube-api-access-crm22" (OuterVolumeSpecName: "kube-api-access-crm22") pod "bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" (UID: "bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72"). InnerVolumeSpecName "kube-api-access-crm22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:13:47.094830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.094731    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crm22\" (UniqueName: \"kubernetes.io/projected/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-kube-api-access-crm22\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:47.094830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.094768    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:47.094830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.094778    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:47.094830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.094787    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcnxm\" (UniqueName: \"kubernetes.io/projected/ead24e1a-66f3-460c-8f3c-cb25c6afe89a-kube-api-access-xcnxm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:47.158849 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.158808    2569 generic.go:356] "Generic (PLEG): container finished" podID="ead24e1a-66f3-460c-8f3c-cb25c6afe89a" containerID="d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672" exitCode=0
Aug 04 16:13:47.159053 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.158875    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"
Aug 04 16:13:47.159053 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.158894    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b" event={"ID":"ead24e1a-66f3-460c-8f3c-cb25c6afe89a","Type":"ContainerDied","Data":"d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672"}
Aug 04 16:13:47.159053 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.158958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b" event={"ID":"ead24e1a-66f3-460c-8f3c-cb25c6afe89a","Type":"ContainerDied","Data":"916ebc4c8d3a0f14842423743e0974f3ff848e2bae9e487974a37b5099a92e30"}
Aug 04 16:13:47.159053 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.158981    2569 scope.go:117] "RemoveContainer" containerID="d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672"
Aug 04 16:13:47.160346 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.160322    2569 generic.go:356] "Generic (PLEG): container finished" podID="bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" containerID="31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232" exitCode=0
Aug 04 16:13:47.160488 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.160378    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"
Aug 04 16:13:47.160488 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.160393    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw" event={"ID":"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72","Type":"ContainerDied","Data":"31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232"}
Aug 04 16:13:47.160488 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.160447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw" event={"ID":"bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72","Type":"ContainerDied","Data":"18a2a3ee9f5ead8bd3dc024431d396b108ab42df11e5a49b77cf4864671652b1"}
Aug 04 16:13:47.169612 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.169483    2569 scope.go:117] "RemoveContainer" containerID="d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672"
Aug 04 16:13:47.169907 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:13:47.169885    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672\": container with ID starting with d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672 not found: ID does not exist" containerID="d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672"
Aug 04 16:13:47.170028 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.169919    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672"} err="failed to get container status \"d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672\": rpc error: code = NotFound desc = could not find container \"d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672\": container with ID starting with d675caba86f1afe92839deae23cda537390f73bda52f16a767f12b0b35061672 not found: ID does not exist"
Aug 04 16:13:47.170028 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.169942    2569 scope.go:117] "RemoveContainer" containerID="31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232"
Aug 04 16:13:47.178726 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.178701    2569 scope.go:117] "RemoveContainer" containerID="31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232"
Aug 04 16:13:47.179055 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:13:47.179031    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232\": container with ID starting with 31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232 not found: ID does not exist" containerID="31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232"
Aug 04 16:13:47.179144 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.179064    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232"} err="failed to get container status \"31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232\": rpc error: code = NotFound desc = could not find container \"31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232\": container with ID starting with 31b28ad080776d89548fa6ef195b696f1a2b97097b134c2f264db4065a4ee232 not found: ID does not exist"
Aug 04 16:13:47.221580 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.221539    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"]
Aug 04 16:13:47.235697 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.235657    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-67a15f-5f6bd98656-nwxtw"]
Aug 04 16:13:47.275214 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.275175    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"]
Aug 04 16:13:47.283437 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.283409    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-67a15f-666d5bb6bb-9rf6b"]
Aug 04 16:13:47.965668 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.965611    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a23a29-fbee-4609-8c8c-dc0da38c5e40" path="/var/lib/kubelet/pods/b5a23a29-fbee-4609-8c8c-dc0da38c5e40/volumes"
Aug 04 16:13:47.966159 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.966142    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" path="/var/lib/kubelet/pods/bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72/volumes"
Aug 04 16:13:47.966592 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:47.966578    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ead24e1a-66f3-460c-8f3c-cb25c6afe89a" path="/var/lib/kubelet/pods/ead24e1a-66f3-460c-8f3c-cb25c6afe89a/volumes"
Aug 04 16:13:48.947869 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:48.947825    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:13:54.779371 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779286    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"]
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779616    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" containerName="payload-pre-processing"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779626    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" containerName="payload-pre-processing"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779657    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ead24e1a-66f3-460c-8f3c-cb25c6afe89a" containerName="payload-processing"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779663    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ead24e1a-66f3-460c-8f3c-cb25c6afe89a" containerName="payload-processing"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779671    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a23a29-fbee-4609-8c8c-dc0da38c5e40" containerName="revoke-keys"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779676    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a23a29-fbee-4609-8c8c-dc0da38c5e40" containerName="revoke-keys"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779734    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a23a29-fbee-4609-8c8c-dc0da38c5e40" containerName="revoke-keys"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779742    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ead24e1a-66f3-460c-8f3c-cb25c6afe89a" containerName="payload-processing"
Aug 04 16:13:54.779774 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.779750    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbbbc0ac-c1b7-4d70-a5ee-d675d810bd72" containerName="payload-pre-processing"
Aug 04 16:13:54.784158 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.784139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:54.793911 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.793834    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Aug 04 16:13:54.794437 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.794413    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2g8l5\""
Aug 04 16:13:54.795985 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.795955    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"]
Aug 04 16:13:54.859023 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.858990    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/460a3950-e13a-455c-ac36-c0b73e3bbb74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn\" (UID: \"460a3950-e13a-455c-ac36-c0b73e3bbb74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:54.959983 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.959939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/460a3950-e13a-455c-ac36-c0b73e3bbb74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn\" (UID: \"460a3950-e13a-455c-ac36-c0b73e3bbb74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:54.960517 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:54.960497    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/460a3950-e13a-455c-ac36-c0b73e3bbb74-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn\" (UID: \"460a3950-e13a-455c-ac36-c0b73e3bbb74\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:55.094249 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:55.094137    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:55.248259 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:55.248097    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"]
Aug 04 16:13:55.251102 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:13:55.251072    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460a3950_e13a_455c_ac36_c0b73e3bbb74.slice/crio-c151ef7aaf76d209853368a03e39b19ba44926a931e422b9531e395f6dc01d95 WatchSource:0}: Error finding container c151ef7aaf76d209853368a03e39b19ba44926a931e422b9531e395f6dc01d95: Status 404 returned error can't find the container with id c151ef7aaf76d209853368a03e39b19ba44926a931e422b9531e395f6dc01d95
Aug 04 16:13:56.209127 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:56.209027    2569 generic.go:356] "Generic (PLEG): container finished" podID="460a3950-e13a-455c-ac36-c0b73e3bbb74" containerID="d5fe92c7fa2877d6890506c89fcea7073c59f4633989c36b53c0f53fdc6d2afd" exitCode=0
Aug 04 16:13:56.209127 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:56.209108    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn" event={"ID":"460a3950-e13a-455c-ac36-c0b73e3bbb74","Type":"ContainerDied","Data":"d5fe92c7fa2877d6890506c89fcea7073c59f4633989c36b53c0f53fdc6d2afd"}
Aug 04 16:13:56.209579 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:56.209161    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn" event={"ID":"460a3950-e13a-455c-ac36-c0b73e3bbb74","Type":"ContainerStarted","Data":"c151ef7aaf76d209853368a03e39b19ba44926a931e422b9531e395f6dc01d95"}
Aug 04 16:13:57.349878 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:57.349849    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:57.486961 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:57.486860    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/460a3950-e13a-455c-ac36-c0b73e3bbb74-maas-api-service-ca\") pod \"460a3950-e13a-455c-ac36-c0b73e3bbb74\" (UID: \"460a3950-e13a-455c-ac36-c0b73e3bbb74\") "
Aug 04 16:13:57.487263 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:57.487237    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/460a3950-e13a-455c-ac36-c0b73e3bbb74-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "460a3950-e13a-455c-ac36-c0b73e3bbb74" (UID: "460a3950-e13a-455c-ac36-c0b73e3bbb74"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:13:57.587878 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:57.587834    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/460a3950-e13a-455c-ac36-c0b73e3bbb74-maas-api-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:13:58.219713 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:58.219679    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"
Aug 04 16:13:58.219875 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:58.219687    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn" event={"ID":"460a3950-e13a-455c-ac36-c0b73e3bbb74","Type":"ContainerDied","Data":"c151ef7aaf76d209853368a03e39b19ba44926a931e422b9531e395f6dc01d95"}
Aug 04 16:13:58.219875 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:13:58.219796    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c151ef7aaf76d209853368a03e39b19ba44926a931e422b9531e395f6dc01d95"
Aug 04 16:14:03.625701 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.625660    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"]
Aug 04 16:14:03.630277 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.630245    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-4d9499-29764320-x429n"]
Aug 04 16:14:03.700905 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.700869    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"]
Aug 04 16:14:03.701137 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.701090    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9" podUID="df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" containerName="payload-processing" containerID="cri-o://991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b" gracePeriod=30
Aug 04 16:14:03.709700 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.709623    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"]
Aug 04 16:14:03.710074 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.710009    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67" podUID="23224110-c292-4a4f-98d0-8774aa22c384" containerName="payload-pre-processing" containerID="cri-o://b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57" gracePeriod=30
Aug 04 16:14:03.913164 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.913120    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"]
Aug 04 16:14:03.914496 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.914465    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-4d9499-rb7pn"]
Aug 04 16:14:03.975488 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.975444    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="448c0fd5-1af7-4d55-aa19-10dccf1a3f44" path="/var/lib/kubelet/pods/448c0fd5-1af7-4d55-aa19-10dccf1a3f44/volumes"
Aug 04 16:14:03.975983 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.975960    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="460a3950-e13a-455c-ac36-c0b73e3bbb74" path="/var/lib/kubelet/pods/460a3950-e13a-455c-ac36-c0b73e3bbb74/volumes"
Aug 04 16:14:03.994755 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.994733    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 16:14:03.996836 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:03.996818    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 16:14:04.053627 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.053536    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8vkm\" (UniqueName: \"kubernetes.io/projected/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-kube-api-access-m8vkm\") pod \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") "
Aug 04 16:14:04.053830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.053679    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-plugins-config-volume\") pod \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\" (UID: \"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc\") "
Aug 04 16:14:04.053830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.053713    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmjs2\" (UniqueName: \"kubernetes.io/projected/23224110-c292-4a4f-98d0-8774aa22c384-kube-api-access-cmjs2\") pod \"23224110-c292-4a4f-98d0-8774aa22c384\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") "
Aug 04 16:14:04.053830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.053740    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23224110-c292-4a4f-98d0-8774aa22c384-plugins-config-volume\") pod \"23224110-c292-4a4f-98d0-8774aa22c384\" (UID: \"23224110-c292-4a4f-98d0-8774aa22c384\") "
Aug 04 16:14:04.054171 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.054139    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" (UID: "df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:14:04.054332 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.054181    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23224110-c292-4a4f-98d0-8774aa22c384-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23224110-c292-4a4f-98d0-8774aa22c384" (UID: "23224110-c292-4a4f-98d0-8774aa22c384"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 04 16:14:04.056316 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.056289    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23224110-c292-4a4f-98d0-8774aa22c384-kube-api-access-cmjs2" (OuterVolumeSpecName: "kube-api-access-cmjs2") pod "23224110-c292-4a4f-98d0-8774aa22c384" (UID: "23224110-c292-4a4f-98d0-8774aa22c384"). InnerVolumeSpecName "kube-api-access-cmjs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:14:04.056440 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.056312    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-kube-api-access-m8vkm" (OuterVolumeSpecName: "kube-api-access-m8vkm") pod "df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" (UID: "df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc"). InnerVolumeSpecName "kube-api-access-m8vkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 04 16:14:04.155104 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.155004    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8vkm\" (UniqueName: \"kubernetes.io/projected/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-kube-api-access-m8vkm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:14:04.155104 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.155038    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:14:04.155104 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.155048    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmjs2\" (UniqueName: \"kubernetes.io/projected/23224110-c292-4a4f-98d0-8774aa22c384-kube-api-access-cmjs2\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:14:04.155104 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.155057    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23224110-c292-4a4f-98d0-8774aa22c384-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Aug 04 16:14:04.251034 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.250999    2569 generic.go:356] "Generic (PLEG): container finished" podID="df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" containerID="991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b" exitCode=0
Aug 04 16:14:04.251219 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.251047    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"
Aug 04 16:14:04.251219 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.251085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9" event={"ID":"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc","Type":"ContainerDied","Data":"991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b"}
Aug 04 16:14:04.251219 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.251171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9" event={"ID":"df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc","Type":"ContainerDied","Data":"9838ceba07a0b028eef560fb26ceead73228a54cdaeee28adcae4983ec575d21"}
Aug 04 16:14:04.251219 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.251200    2569 scope.go:117] "RemoveContainer" containerID="991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b"
Aug 04 16:14:04.252581 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.252559    2569 generic.go:356] "Generic (PLEG): container finished" podID="23224110-c292-4a4f-98d0-8774aa22c384" containerID="b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57" exitCode=0
Aug 04 16:14:04.252742 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.252612    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"
Aug 04 16:14:04.252742 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.252617    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67" event={"ID":"23224110-c292-4a4f-98d0-8774aa22c384","Type":"ContainerDied","Data":"b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57"}
Aug 04 16:14:04.252742 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.252713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67" event={"ID":"23224110-c292-4a4f-98d0-8774aa22c384","Type":"ContainerDied","Data":"353af99a2c6aaa4f6364a67ea2fd6e9a7a1a8d7d0610ebfa38f53ca7635ee6be"}
Aug 04 16:14:04.260219 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.260199    2569 scope.go:117] "RemoveContainer" containerID="991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b"
Aug 04 16:14:04.260575 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:14:04.260557    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b\": container with ID starting with 991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b not found: ID does not exist" containerID="991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b"
Aug 04 16:14:04.260668 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.260590    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b"} err="failed to get container status \"991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b\": rpc error: code = NotFound desc = could not find container \"991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b\": container with ID starting with 991bd4d88e0a7ce4c3940f01d5f03cd6fed31c16e823a007b230cc583039d76b not found: ID does not exist"
Aug 04 16:14:04.260668 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.260614    2569 scope.go:117] "RemoveContainer" containerID="b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57"
Aug 04 16:14:04.269085 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.269061    2569 scope.go:117] "RemoveContainer" containerID="b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57"
Aug 04 16:14:04.269404 ip-10-0-136-40 kubenswrapper[2569]: E0804 16:14:04.269385    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57\": container with ID starting with b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57 not found: ID does not exist" containerID="b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57"
Aug 04 16:14:04.269456 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.269414    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57"} err="failed to get container status \"b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57\": rpc error: code = NotFound desc = could not find container \"b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57\": container with ID starting with b6945156dbe5972eae6718f0311bf99642789fbfa9feae76a192e2308a956e57 not found: ID does not exist"
Aug 04 16:14:04.279047 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.279020    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"]
Aug 04 16:14:04.282718 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.282691    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-4d9499-548cd464c6-wxj67"]
Aug 04 16:14:04.296449 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.296418    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"]
Aug 04 16:14:04.303180 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:04.303153    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-4d9499-67d87cf7d6-mxcp9"]
Aug 04 16:14:05.827684 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:05.827649    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-l7x4t"]
Aug 04 16:14:05.964506 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:05.964470    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23224110-c292-4a4f-98d0-8774aa22c384" path="/var/lib/kubelet/pods/23224110-c292-4a4f-98d0-8774aa22c384/volumes"
Aug 04 16:14:05.964939 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:05.964923    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" path="/var/lib/kubelet/pods/df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc/volumes"
Aug 04 16:14:17.553308 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:17.553271    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7gpd8_f86e84a8-cf58-42e0-8467-ebcf739b4750/manager/0.log"
Aug 04 16:14:18.079863 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:18.079826    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77c9c696b6-5xgqz_99821cb7-a349-4786-8d61-9437543e6269/copy-manifests/0.log"
Aug 04 16:14:18.093044 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:18.093006    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77c9c696b6-5xgqz_99821cb7-a349-4786-8d61-9437543e6269/manager/0.log"
Aug 04 16:14:20.487725 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:20.487696    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-fwn7r_765f70f7-988d-464f-a86a-28e7bd35b0cd/kuadrant-console-plugin/0.log"
Aug 04 16:14:20.734092 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:20.734067    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-l7x4t_b420a609-7113-4bcc-8a37-44f8ac15c8bd/limitador/0.log"
Aug 04 16:14:21.585343 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:21.585315    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86b9dff769-xlmdx_bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7/kube-auth-proxy/0.log"
Aug 04 16:14:23.024188 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:23.024162    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-pssvn_db23bef1-ac30-4f71-b06d-08346751fed8/main/0.log"
Aug 04 16:14:29.207807 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.207774    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qtx9z/must-gather-nc629"]
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208097    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460a3950-e13a-455c-ac36-c0b73e3bbb74" containerName="revoke-keys"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208108    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="460a3950-e13a-455c-ac36-c0b73e3bbb74" containerName="revoke-keys"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208122    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" containerName="payload-processing"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208128    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" containerName="payload-processing"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208137    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23224110-c292-4a4f-98d0-8774aa22c384" containerName="payload-pre-processing"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208144    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="23224110-c292-4a4f-98d0-8774aa22c384" containerName="payload-pre-processing"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208196    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="460a3950-e13a-455c-ac36-c0b73e3bbb74" containerName="revoke-keys"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208207    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="df1c0a4f-3d0a-49f0-bd9d-96e5b11105bc" containerName="payload-processing"
Aug 04 16:14:29.208212 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.208213    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="23224110-c292-4a4f-98d0-8774aa22c384" containerName="payload-pre-processing"
Aug 04 16:14:29.211289 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.211271    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.214684 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.214660    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qtx9z\"/\"kube-root-ca.crt\""
Aug 04 16:14:29.214940 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.214923    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qtx9z\"/\"openshift-service-ca.crt\""
Aug 04 16:14:29.215723 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.215705    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qtx9z\"/\"default-dockercfg-65jnv\""
Aug 04 16:14:29.230830 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.230799    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qtx9z/must-gather-nc629"]
Aug 04 16:14:29.280591 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.280551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d9aa4210-e42d-4b16-b272-16bc6804fd55-must-gather-output\") pod \"must-gather-nc629\" (UID: \"d9aa4210-e42d-4b16-b272-16bc6804fd55\") " pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.280591 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.280600    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5tgq\" (UniqueName: \"kubernetes.io/projected/d9aa4210-e42d-4b16-b272-16bc6804fd55-kube-api-access-r5tgq\") pod \"must-gather-nc629\" (UID: \"d9aa4210-e42d-4b16-b272-16bc6804fd55\") " pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.381479 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.381448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d9aa4210-e42d-4b16-b272-16bc6804fd55-must-gather-output\") pod \"must-gather-nc629\" (UID: \"d9aa4210-e42d-4b16-b272-16bc6804fd55\") " pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.381692 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.381487    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5tgq\" (UniqueName: \"kubernetes.io/projected/d9aa4210-e42d-4b16-b272-16bc6804fd55-kube-api-access-r5tgq\") pod \"must-gather-nc629\" (UID: \"d9aa4210-e42d-4b16-b272-16bc6804fd55\") " pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.381847 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.381824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d9aa4210-e42d-4b16-b272-16bc6804fd55-must-gather-output\") pod \"must-gather-nc629\" (UID: \"d9aa4210-e42d-4b16-b272-16bc6804fd55\") " pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.397685 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.397630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5tgq\" (UniqueName: \"kubernetes.io/projected/d9aa4210-e42d-4b16-b272-16bc6804fd55-kube-api-access-r5tgq\") pod \"must-gather-nc629\" (UID: \"d9aa4210-e42d-4b16-b272-16bc6804fd55\") " pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.520478 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.520387    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qtx9z/must-gather-nc629"
Aug 04 16:14:29.675208 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.675170    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qtx9z/must-gather-nc629"]
Aug 04 16:14:29.678565 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:14:29.678538    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9aa4210_e42d_4b16_b272_16bc6804fd55.slice/crio-ca70fb326e498a8389911d46ca3249a09ffa99d23293d28cbbde00f73cf24f93 WatchSource:0}: Error finding container ca70fb326e498a8389911d46ca3249a09ffa99d23293d28cbbde00f73cf24f93: Status 404 returned error can't find the container with id ca70fb326e498a8389911d46ca3249a09ffa99d23293d28cbbde00f73cf24f93
Aug 04 16:14:29.680861 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:29.680842    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 04 16:14:30.388278 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:30.388238    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qtx9z/must-gather-nc629" event={"ID":"d9aa4210-e42d-4b16-b272-16bc6804fd55","Type":"ContainerStarted","Data":"ca70fb326e498a8389911d46ca3249a09ffa99d23293d28cbbde00f73cf24f93"}
Aug 04 16:14:32.404174 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:32.404034    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qtx9z/must-gather-nc629" event={"ID":"d9aa4210-e42d-4b16-b272-16bc6804fd55","Type":"ContainerStarted","Data":"c2ba14177a7b93622e70c51ec2d2478a443a4c4c7f5d80c678e84aa5bb8ce417"}
Aug 04 16:14:32.404174 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:32.404093    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qtx9z/must-gather-nc629" event={"ID":"d9aa4210-e42d-4b16-b272-16bc6804fd55","Type":"ContainerStarted","Data":"7b7a949fcb45a3705dace7c3b284c3ef24a07356a222b8a627dff4a39e300f2c"}
Aug 04 16:14:32.438849 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:32.438787    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qtx9z/must-gather-nc629" podStartSLOduration=0.967030413 podStartE2EDuration="3.438768209s" totalImagesPullingTime="2.471737796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:14:32.437894772 +0000 UTC m=+3814.944192829" watchObservedRunningTime="2026-08-04 16:14:32.438768209 +0000 UTC m=+3814.945066267"
Aug 04 16:14:34.355825 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:34.355763    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bzd8c_319e5610-9a08-4abb-b819-a1d286392a65/global-pull-secret-syncer/0.log"
Aug 04 16:14:34.812881 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:34.812852    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zcvxt_4a87dcde-1842-4be7-884a-ab63c3ea6d25/konnectivity-agent/0.log"
Aug 04 16:14:34.902109 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:34.902075    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-40.ec2.internal_21665e5cbddb8060242764c579d30712/haproxy/0.log"
Aug 04 16:14:39.470579 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:39.470549    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-fwn7r_765f70f7-988d-464f-a86a-28e7bd35b0cd/kuadrant-console-plugin/0.log"
Aug 04 16:14:39.639126 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:39.639098    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-l7x4t_b420a609-7113-4bcc-8a37-44f8ac15c8bd/limitador/0.log"
Aug 04 16:14:41.331036 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.331000    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/alertmanager/0.log"
Aug 04 16:14:41.352856 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.352824    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/config-reloader/0.log"
Aug 04 16:14:41.380961 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.380931    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/kube-rbac-proxy-web/0.log"
Aug 04 16:14:41.405157 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.405124    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/kube-rbac-proxy/0.log"
Aug 04 16:14:41.443572 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.443540    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/kube-rbac-proxy-metric/0.log"
Aug 04 16:14:41.473044 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.471590    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/prom-label-proxy/0.log"
Aug 04 16:14:41.509393 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.509342    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b4dcbf92-c560-49b9-b93e-a64401b60761/init-config-reloader/0.log"
Aug 04 16:14:41.615383 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.615280    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-2qrq8_b1e8a259-7a35-4a92-8847-14d17a528111/kube-state-metrics/0.log"
Aug 04 16:14:41.646347 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.646125    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-2qrq8_b1e8a259-7a35-4a92-8847-14d17a528111/kube-rbac-proxy-main/0.log"
Aug 04 16:14:41.690195 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.690158    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-2qrq8_b1e8a259-7a35-4a92-8847-14d17a528111/kube-rbac-proxy-self/0.log"
Aug 04 16:14:41.852148 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.852112    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7cwbk_ab61273d-1d70-4fa6-8205-3d5ce36a8744/node-exporter/0.log"
Aug 04 16:14:41.922338 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.922241    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7cwbk_ab61273d-1d70-4fa6-8205-3d5ce36a8744/kube-rbac-proxy/0.log"
Aug 04 16:14:41.959535 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:41.959383    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7cwbk_ab61273d-1d70-4fa6-8205-3d5ce36a8744/init-textfile/0.log"
Aug 04 16:14:42.311439 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.311398    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/prometheus/0.log"
Aug 04 16:14:42.348009 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.347980    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/config-reloader/0.log"
Aug 04 16:14:42.380457 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.380429    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/thanos-sidecar/0.log"
Aug 04 16:14:42.409987 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.409912    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/kube-rbac-proxy-web/0.log"
Aug 04 16:14:42.442279 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.442255    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/kube-rbac-proxy/0.log"
Aug 04 16:14:42.468523 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.468487    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/kube-rbac-proxy-thanos/0.log"
Aug 04 16:14:42.496996 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:42.496967    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f17170f9-3c2d-49b6-b09f-f81fff5c8d65/init-config-reloader/0.log"
Aug 04 16:14:44.521792 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.521761    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"]
Aug 04 16:14:44.529090 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.529056    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.542052 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.541568    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"]
Aug 04 16:14:44.560671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.559108    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rb6z\" (UniqueName: \"kubernetes.io/projected/0c7ec53f-76b3-4b80-b6a7-7985e1146817-kube-api-access-2rb6z\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.560671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.559162    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-podres\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.560671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.559213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-sys\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.560671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.559240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-lib-modules\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.560671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.559308    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-proc\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.660338 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.660295    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rb6z\" (UniqueName: \"kubernetes.io/projected/0c7ec53f-76b3-4b80-b6a7-7985e1146817-kube-api-access-2rb6z\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.660607 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.660591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-podres\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.661290 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.661266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-sys\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.661490 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.661468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-lib-modules\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.661710 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.661606    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-lib-modules\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.661710 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.660858    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-podres\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.661710 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.661467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-sys\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.661710 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.661673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-proc\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.662030 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.661738    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c7ec53f-76b3-4b80-b6a7-7985e1146817-proc\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.689093 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.689062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rb6z\" (UniqueName: \"kubernetes.io/projected/0c7ec53f-76b3-4b80-b6a7-7985e1146817-kube-api-access-2rb6z\") pod \"perf-node-gather-daemonset-wlkgr\" (UID: \"0c7ec53f-76b3-4b80-b6a7-7985e1146817\") " pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:44.843992 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:44.843902    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:45.052671 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:45.052603    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"]
Aug 04 16:14:45.057435 ip-10-0-136-40 kubenswrapper[2569]: W0804 16:14:45.057383    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c7ec53f_76b3_4b80_b6a7_7985e1146817.slice/crio-cae3d75a4a5fa99676376b5878c09b31736d609c05f09aa4c45f3cc03ab9e281 WatchSource:0}: Error finding container cae3d75a4a5fa99676376b5878c09b31736d609c05f09aa4c45f3cc03ab9e281: Status 404 returned error can't find the container with id cae3d75a4a5fa99676376b5878c09b31736d609c05f09aa4c45f3cc03ab9e281
Aug 04 16:14:45.495222 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:45.495131    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74cc45dc7-qbb88_516efb51-fbb4-4d41-b9f5-5cbbc756749c/console/0.log"
Aug 04 16:14:45.505162 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:45.505118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr" event={"ID":"0c7ec53f-76b3-4b80-b6a7-7985e1146817","Type":"ContainerStarted","Data":"b5b085fde0652b05fa4eb3da6ba3e0ac3c7a2d4bd40ccf3e7186f5180471bd24"}
Aug 04 16:14:45.505348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:45.505182    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr" event={"ID":"0c7ec53f-76b3-4b80-b6a7-7985e1146817","Type":"ContainerStarted","Data":"cae3d75a4a5fa99676376b5878c09b31736d609c05f09aa4c45f3cc03ab9e281"}
Aug 04 16:14:45.505348 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:45.505256    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:45.523933 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:45.523865    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr" podStartSLOduration=1.5238440629999999 podStartE2EDuration="1.523844063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 16:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:14:45.52314383 +0000 UTC m=+3828.029441888" watchObservedRunningTime="2026-08-04 16:14:45.523844063 +0000 UTC m=+3828.030142118"
Aug 04 16:14:47.528311 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:47.528275    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5k478_45fa4203-4fe3-4862-a527-401fb126b252/dns/0.log"
Aug 04 16:14:47.555172 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:47.555147    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5k478_45fa4203-4fe3-4862-a527-401fb126b252/kube-rbac-proxy/0.log"
Aug 04 16:14:47.672661 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:47.672602    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xmz4_147f4ebf-ef58-4c19-a0df-4948c85d78f7/dns-node-resolver/0.log"
Aug 04 16:14:48.316111 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:48.316069    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-78c985b499-qf2qg_6f8658b5-ba77-48f1-bb28-79360fe4e918/registry/0.log"
Aug 04 16:14:48.343875 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:48.343845    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kgs4p_a16acf50-04d8-4549-9350-2061249f53c5/node-ca/0.log"
Aug 04 16:14:49.678177 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:49.678149    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86b9dff769-xlmdx_bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7/kube-auth-proxy/0.log"
Aug 04 16:14:50.766879 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:50.766845    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cncqn_0f24288a-aa4e-4462-ae64-8cbdbd8c5c42/serve-healthcheck-canary/0.log"
Aug 04 16:14:51.492817 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:51.492788    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8h4jc_7215b5c0-7b9b-476e-8f79-e46eac80dbca/kube-rbac-proxy/0.log"
Aug 04 16:14:51.518199 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:51.518166    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8h4jc_7215b5c0-7b9b-476e-8f79-e46eac80dbca/exporter/0.log"
Aug 04 16:14:51.526488 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:51.526461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qtx9z/perf-node-gather-daemonset-wlkgr"
Aug 04 16:14:51.545396 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:51.545345    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8h4jc_7215b5c0-7b9b-476e-8f79-e46eac80dbca/extractor/0.log"
Aug 04 16:14:54.585091 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:54.585063    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7gpd8_f86e84a8-cf58-42e0-8467-ebcf739b4750/manager/0.log"
Aug 04 16:14:55.156191 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:55.156123    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77c9c696b6-5xgqz_99821cb7-a349-4786-8d61-9437543e6269/manager/0.log"
Aug 04 16:14:55.176448 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:55.176419    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77c9c696b6-5xgqz_99821cb7-a349-4786-8d61-9437543e6269/copy-manifests/0.log"
Aug 04 16:14:55.734292 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:55.734259    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86b9dff769-xlmdx_bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7/kube-auth-proxy/0.log"
Aug 04 16:14:56.709916 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:56.709827    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59d7b7db79-wh7nn_74b5ab48-00f7-4391-830c-f3e3bc5fbddc/manager/0.log"
Aug 04 16:14:57.585831 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:57.585804    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86b9dff769-xlmdx_bcbca683-3d5c-48b4-8a68-b1ca3a5dc5e7/kube-auth-proxy/0.log"
Aug 04 16:14:58.534841 ip-10-0-136-40 kubenswrapper[2569]: I0804 16:14:58.534802    2569 scope.go:117] "RemoveContainer" containerID="56f8134068642af7470668aa08208df3d06659f972791409e9a5de8c08a32327"