Jul 17 09:24:54.221995 ip-10-0-141-244 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 09:24:54.222011 ip-10-0-141-244 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 09:24:54.222022 ip-10-0-141-244 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 09:24:54.222362 ip-10-0-141-244 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 09:25:04.333926 ip-10-0-141-244 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 09:25:04.333945 ip-10-0-141-244 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 865ac876718c424b908343cb55386313 --
Jul 17 09:26:55.863181 ip-10-0-141-244 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 09:26:56.264630 ip-10-0-141-244 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:26:56.264630 ip-10-0-141-244 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 09:26:56.264630 ip-10-0-141-244 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:26:56.264630 ip-10-0-141-244 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 09:26:56.264630 ip-10-0-141-244 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:26:56.265340 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.264699    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 09:26:56.269169 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269141    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:56.269169 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269165    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:56.269169 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269172    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:56.269169 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269177    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269181    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269184    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269187    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269190    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269193    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269195    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269199    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269203    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269206    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269215    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269218    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269221    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269223    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269226    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269229    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269231    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269234    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269236    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:56.269334 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269239    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269241    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269243    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269246    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269248    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269251    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269253    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269256    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269258    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269261    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269263    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269266    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269268    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269271    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269273    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269276    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269279    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269282    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269286    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269288    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:56.269802 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269292    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269294    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269297    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269299    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269302    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269304    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269307    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269309    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269311    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269314    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269316    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269319    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269321    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269323    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269326    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269328    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269333    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269336    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269339    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:56.270315 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269342    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269345    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269348    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269351    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269353    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269356    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269358    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269361    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269365    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269368    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269371    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269374    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269378    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269380    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269383    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269386    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269388    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269390    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269393    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:56.270833 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269395    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269398    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269400    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269403    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269405    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269408    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269410    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269413    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269416    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269418    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269422    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269424    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269426    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269429    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269431    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269434    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269436    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269439    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.269441    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270239    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:56.271302 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270247    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270251    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270254    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270257    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270259    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270262    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270265    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270267    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270270    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270273    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270276    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270279    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270281    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270284    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270286    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270289    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270291    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270293    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270296    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270298    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:56.271792 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270302    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270306    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270309    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270312    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270314    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270317    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270320    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270323    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270325    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270328    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270330    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270332    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270335    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270338    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270341    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270344    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270347    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270352    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270356    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270358    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:56.272307 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270361    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270363    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270366    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270369    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270371    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270374    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270376    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270378    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270381    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270384    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270386    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270389    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270391    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270393    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270396    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270398    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270401    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270404    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270406    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270409    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:56.272788 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270411    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270414    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270416    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270418    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270421    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270423    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270426    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270429    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270431    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270434    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270437    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270439    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270442    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270444    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270447    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270449    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270453    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270455    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270457    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270460    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:56.273312 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270462    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270465    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270467    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270470    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270472    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270475    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270477    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270480    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270482    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270485    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270489    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270492    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270495    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270498    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270500    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270502    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270505    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270508    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270932    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:56.273883 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270938    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270941    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270944    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270948    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270951    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270954    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270956    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270959    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270961    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270964    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270966    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270969    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270971    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270974    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270976    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270978    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270981    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270983    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270986    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:56.274344 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270989    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270993    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270996    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.270999    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271002    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271004    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271007    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271009    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271012    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271014    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271021    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271024    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271026    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271029    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271031    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271034    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271037    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271041    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271043    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:56.274798 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271046    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271048    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271051    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271054    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271056    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271059    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271063    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271068    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271071    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271073    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271076    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271078    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271080    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271083    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271085    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271088    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271092    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271095    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271098    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271101    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:56.275288 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271103    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271106    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271109    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271113    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271116    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271119    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271121    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271124    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271127    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271129    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271132    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271135    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271138    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271140    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271143    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271145    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271148    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271150    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271152    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271156    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:56.275846 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271159    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271161    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271164    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271166    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271168    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271171    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271173    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271175    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271178    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271180    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271182    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271185    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271187    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271190    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271192    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271196    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271198    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271201    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271203    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271205    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:56.276338 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271280    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271288    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271294    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271299    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271304    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271308    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271312    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271316    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271320    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271322    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271326    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271331    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271334    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271337    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271340    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271343    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271345    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271348    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271353    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271356    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271359    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271362    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271366    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271369    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 09:26:56.276852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271372    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271375    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271378    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271383    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271386    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271389    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271392    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271395    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271399    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271402    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271405    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271408    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271410    2556 flags.go:64] FLAG: --enable-server="true"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271414    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271418    2556 flags.go:64] FLAG: --event-burst="100"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271421    2556 flags.go:64] FLAG: --event-qps="50"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271424    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271427    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271430    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271435    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271437    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271440    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271443    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271446    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271449    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 09:26:56.277430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271452    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271455    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271458    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271461    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271463    2556 flags.go:64] FLAG: --feature-gates=""
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271467    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271470    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271473    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271476    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271479    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271483    2556 flags.go:64] FLAG: --help="false"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271486    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271489    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271492    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271495    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271498    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271502    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271505    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271507    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271510    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271513    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271516    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271519    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271522    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 17 09:26:56.278079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271525    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271528    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271534    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271536    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271539    2556 flags.go:64] FLAG: --lock-file=""
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271541    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271544    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271547    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271552    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271555    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271558    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271560    2556 flags.go:64] FLAG: --logging-format="text"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271563    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271566    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271569    2556 flags.go:64] FLAG: --manifest-url=""
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271572    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271576    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271579    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271585    2556 flags.go:64] FLAG: --max-pods="110"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271587    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271590    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271593    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271596    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271599    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271602    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 09:26:56.278734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271604    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271612    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271615    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271618    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271621    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271624    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271630    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271633    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271636    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271641    2556 flags.go:64] FLAG: --port="10250"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271644    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271646    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ddb0be77ae3214d0"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271649    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271652    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271655    2556 flags.go:64] FLAG: --register-node="true"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271658    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271661    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271664    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271667    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271670    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271673    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271676    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271679    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271682    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271685    2556 flags.go:64] FLAG: --runonce="false"
Jul 17 09:26:56.279416 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271687    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271692    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271695    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271698    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271700    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271703    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271706    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271709    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271712    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271715    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271718    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271721    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271724    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271727    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271730    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271735    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271739    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271742    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271746    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271749    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271752    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271754    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271757    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271760    2556 flags.go:64] FLAG: --v="2"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271764    2556 flags.go:64] FLAG: --version="false"
Jul 17 09:26:56.280069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271768    2556 flags.go:64] FLAG: --vmodule=""
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271771    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.271774    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271904    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271909    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271912    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271915    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271918    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271922    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271925    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271928    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271931    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271933    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271936    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271938    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271941    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271944    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271946    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271948    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271951    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:56.280678 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271954    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271957    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271960    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271964    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271966    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271969    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271973    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271976    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271979    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271982    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271985    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271988    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271991    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271993    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271996    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.271999    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272001    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272004    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272006    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:56.281197 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272009    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272013    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272017    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272020    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272022    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272025    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272027    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272030    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272032    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272035    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272037    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272040    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272042    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272044    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272047    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272050    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272054    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272057    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272059    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:56.281683 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272062    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272064    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272067    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272070    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272072    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272075    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272077    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272080    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272082    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272084    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272087    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272089    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272091    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272094    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272098    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272100    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272103    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272105    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272108    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272112    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:56.282163 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272115    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272118    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272121    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272124    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272127    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272129    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272132    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272135    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272138    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272143    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272145    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272148    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272150    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272153    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272155    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272158    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272160    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272163    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272165    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272167    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:56.282662 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272170    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272172    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272175    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.272177    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.272855    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.280042    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.280061    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.280082    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.280089    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280149    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280156    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280159    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280162    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280165    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:56.283156 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280168    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280170    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280174    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280177    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280180    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280182    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280185    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280187    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280190    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280192    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280195    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280197    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280200    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280202    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280205    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280208    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280210    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280213    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280217    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:56.283553 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280221    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280224    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280227    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280230    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280232    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280236    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280239    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280242    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280244    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280247    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280249    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280252    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280254    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280256    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280259    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280261    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280264    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280266    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280268    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280271    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:56.284035 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280273    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280275    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280278    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280281    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280283    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280287    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280291    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280294    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280296    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280299    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280301    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280304    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280307    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280310    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280312    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280315    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280317    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280320    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280323    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:56.284521 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280326    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280328    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280330    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280333    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280335    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280338    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280340    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280343    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280345    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280348    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280350    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280352    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280355    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280357    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280360    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280362    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280365    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280367    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280369    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280373    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:56.285024 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280376    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280379    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280381    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280384    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280387    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280389    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280392    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280394    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280396    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280399    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280402    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280404    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280408    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280410    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280414    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:56.285525 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280417    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.280423    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280571    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280576    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280579    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280582    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280585    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280587    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280590    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280592    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280595    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280599    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280604    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280607    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280609    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:56.285972 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280611    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280614    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280617    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280619    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280622    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280625    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280627    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280630    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280632    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280635    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280637    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280640    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280642    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280644    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280647    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280651    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280653    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280656    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280658    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280660    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:56.286348 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280663    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280665    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280668    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280671    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280673    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280676    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280678    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280683    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280686    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280690    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280693    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280696    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280699    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280702    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280704    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280707    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280709    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280712    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280714    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:56.286841 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280717    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280719    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280721    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280726    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280729    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280731    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280734    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280737    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280740    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280743    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280745    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280748    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280750    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280753    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280755    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280758    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280761    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280763    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280766    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:56.287551 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280768    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280770    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280773    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280775    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280778    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280780    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280782    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280785    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280787    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280790    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280792    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280795    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280797    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280799    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280802    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280804    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280807    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280809    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280812    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280814    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:56.288065 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280832    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280837    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280840    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280845    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280848    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280850    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280853    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:56.280855    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.280860    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.281003    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.283127    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.284444    2556 server.go:1019] "Starting client certificate rotation"
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.284545    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:26:56.288729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.284618    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:26:56.289291 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.287050    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:26:56.290774 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.290750    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:26:56.301055 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.301027    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 17 09:26:56.306265 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.306240    2556 log.go:25] "Validated CRI v1 image API"
Jul 17 09:26:56.308678 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.308646    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 09:26:56.311177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.311155    2556 fs.go:135] Filesystem UUIDs: map[211f710e-7af7-4399-80dd-1ce8f8ff5883:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 81d9fe82-4284-488f-9f5f-4c30f6ca2266:/dev/nvme0n1p4]
Jul 17 09:26:56.311248 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.311176    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 09:26:56.315659 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.315525    2556 manager.go:216] Machine: {Timestamp:2026-07-17 09:26:56.314994067 +0000 UTC m=+0.325117983 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093079 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec249c595f71c138e9d82efc33760f99 SystemUUID:ec249c59-5f71-c138-e9d8-2efc33760f99 BootID:865ac876-718c-424b-9083-43cb55386313 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:23:a3:26:10:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:23:a3:26:10:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:81:c0:7f:ba:b4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 09:26:56.315659 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.315637    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 09:26:56.315895 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.315729    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 09:26:56.318083 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.318062    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:26:56.318465 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.318432    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 09:26:56.318636 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.318474    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 09:26:56.319766 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.319752    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 09:26:56.319837 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.319772    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 09:26:56.319837 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.319783    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:26:56.320499 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.320485    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:26:56.321722 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.321706    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 09:26:56.325033 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325013    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:26:56.325204 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325191    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 09:26:56.325263 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325233    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 09:26:56.325263 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325250    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 09:26:56.325349 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325267    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 09:26:56.325349 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325282    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 17 09:26:56.325349 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325296    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 09:26:56.325721 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325706    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:26:56.325792 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.325731    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:26:56.328432 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.328413    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 09:26:56.329760 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.329744    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 09:26:56.329841 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.329777    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 09:26:56.330420 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330408    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 09:26:56.330468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330426    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 09:26:56.330468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330438    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 09:26:56.330468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330444    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 09:26:56.330468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330454    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 09:26:56.330468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330461    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 09:26:56.330468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330469    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 09:26:56.330623 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330476    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 09:26:56.330623 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330482    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 09:26:56.330623 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330488    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 09:26:56.330623 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330496    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 09:26:56.330623 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.330503    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 09:26:56.331248 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.331228    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 09:26:56.331248 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.331250    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 09:26:56.335763 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.335736    2556 server.go:1268] "Started kubelet"
Jul 17 09:26:56.336331 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.336072    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 09:26:56.336331 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.336149    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 09:26:56.336331 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.336010    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 09:26:56.336331 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.336220    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q7mww"
Jul 17 09:26:56.338702 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.338680    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 09:26:56.339032 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.338988    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-244.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 09:26:56.339138 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.339029    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 09:26:56.339106 ip-10-0-141-244 systemd[1]: Started Kubernetes Kubelet.
Jul 17 09:26:56.339325 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.339152    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 09:26:56.339806 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.339787    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 09:26:56.341608 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.341585    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 09:26:56.343141 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.343120    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q7mww"
Jul 17 09:26:56.344922 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.344094    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-244.ec2.internal.18c30995cb603096  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-244.ec2.internal,UID:ip-10-0-141-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-244.ec2.internal,},FirstTimestamp:2026-07-17 09:26:56.335704214 +0000 UTC m=+0.345828133,LastTimestamp:2026-07-17 09:26:56.335704214 +0000 UTC m=+0.345828133,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-244.ec2.internal,}"
Jul 17 09:26:56.347597 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.347576    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 09:26:56.347688 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.347652    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 09:26:56.347788 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.347686    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.348550 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348528    2556 factory.go:55] Registering systemd factory
Jul 17 09:26:56.348550 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348553    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 17 09:26:56.348695 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348623    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 09:26:56.348695 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348635    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 09:26:56.348695 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348625    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 09:26:56.348886 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348724    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 09:26:56.348886 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348731    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 09:26:56.348982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348901    2556 factory.go:153] Registering CRI-O factory
Jul 17 09:26:56.348982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348913    2556 factory.go:223] Registration of the crio container factory successfully
Jul 17 09:26:56.348982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348966    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 09:26:56.349125 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.348992    2556 factory.go:103] Registering Raw factory
Jul 17 09:26:56.349125 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.349007    2556 manager.go:1196] Started watching for new ooms in manager
Jul 17 09:26:56.349408 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.349394    2556 manager.go:349] Starting recovery of all containers
Jul 17 09:26:56.351602 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.351584    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:56.355310 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.355100    2556 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-244.ec2.internal\" not found" node="ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.360272 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.360242    2556 manager.go:354] Recovery completed
Jul 17 09:26:56.362781 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.362750    2556 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 17 09:26:56.366019 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.365948    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.368069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368053    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.368141 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368086    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.368141 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368101    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.368618 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368603    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 09:26:56.368618 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368614    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.368699 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368619    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 09:26:56.368699 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.368640    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:26:56.369408 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.369393    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.369458 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.369419    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.369458 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.369428    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.369845 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.369814    2556 policy_none.go:49] "None policy: Start"
Jul 17 09:26:56.369880 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.369850    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 09:26:56.369880 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.369861    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 09:26:56.370966 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.370955    2556 policy_none.go:47] "Start"
Jul 17 09:26:56.402557 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.402529    2556 manager.go:337] "Starting Device Plugin manager"
Jul 17 09:26:56.402727 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.402571    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 09:26:56.402727 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.402583    2556 server.go:87] "Starting device plugin registration server"
Jul 17 09:26:56.402928 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.402910    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 09:26:56.403048 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.402929    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 09:26:56.403154 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.403122    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 09:26:56.403249 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.403228    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 09:26:56.403249 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.403244    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 09:26:56.403785 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.403760    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 09:26:56.403908 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.403809    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.487505 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.487460    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 09:26:56.489225 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.488919    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 09:26:56.489225 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.488958    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 09:26:56.489225 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.488998    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 09:26:56.489225 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.489038    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 09:26:56.491204 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.491183    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:56.503360 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.503326    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.504695 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.504676    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.504839 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.504713    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.504839 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.504727    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.504839 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.504759    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.509925 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.509903    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.510046 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.509933    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-244.ec2.internal\": node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.524477 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.524390    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.589808 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.589771    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"]
Jul 17 09:26:56.589946 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.589864    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.590903 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.590881    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.590963 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.590923    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.590963 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.590938    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.592405 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.592391    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.592528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.592512    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.592580 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.592545    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.593177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.593161    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.593177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.593170    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.593320 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.593195    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.593320 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.593207    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.593320 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.593196    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.593320 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.593268    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.594479 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.594462    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.594479 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.594490    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:56.595266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.595249    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:56.595342 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.595276    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:56.595342 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.595287    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:56.609567 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.609534    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-244.ec2.internal\" not found" node="ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.613838 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.613797    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-244.ec2.internal\" not found" node="ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.624873 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.624842    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.650118 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.650086    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d37f3cbe072cd8ae509ba808ddfa814-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal\" (UID: \"0d37f3cbe072cd8ae509ba808ddfa814\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.650187 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.650120    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2b3895fcc55e8580e43648b59a9ffbae-config\") pod \"kube-apiserver-proxy-ip-10-0-141-244.ec2.internal\" (UID: \"2b3895fcc55e8580e43648b59a9ffbae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.650187 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.650152    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d37f3cbe072cd8ae509ba808ddfa814-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal\" (UID: \"0d37f3cbe072cd8ae509ba808ddfa814\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.725396 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.725365    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.750814 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.750784    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d37f3cbe072cd8ae509ba808ddfa814-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal\" (UID: \"0d37f3cbe072cd8ae509ba808ddfa814\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.750893 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.750836    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d37f3cbe072cd8ae509ba808ddfa814-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal\" (UID: \"0d37f3cbe072cd8ae509ba808ddfa814\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.750893 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.750863    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2b3895fcc55e8580e43648b59a9ffbae-config\") pod \"kube-apiserver-proxy-ip-10-0-141-244.ec2.internal\" (UID: \"2b3895fcc55e8580e43648b59a9ffbae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.750956 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.750891    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d37f3cbe072cd8ae509ba808ddfa814-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal\" (UID: \"0d37f3cbe072cd8ae509ba808ddfa814\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.750956 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.750903    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2b3895fcc55e8580e43648b59a9ffbae-config\") pod \"kube-apiserver-proxy-ip-10-0-141-244.ec2.internal\" (UID: \"2b3895fcc55e8580e43648b59a9ffbae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.750956 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.750913    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d37f3cbe072cd8ae509ba808ddfa814-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal\" (UID: \"0d37f3cbe072cd8ae509ba808ddfa814\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.826140 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.826058    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:56.911720 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.911679    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.918384 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:56.918364    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:56.926183 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:56.926156    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:57.026798 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.026761    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:57.129065 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.129022    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-244.ec2.internal\" not found"
Jul 17 09:26:57.180295 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.180264    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:57.248443 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.248398    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:57.258598 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.258571    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:26:57.260540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.260520    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal"
Jul 17 09:26:57.270839 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.270813    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:26:57.283984 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.283945    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 09:26:57.284137 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.284117    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:26:57.284193 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.284159    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:26:57.284193 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.284174    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:26:57.328655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.328630    2556 apiserver.go:52] "Watching apiserver"
Jul 17 09:26:57.346372 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.346279    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 09:26:57.346990 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.346969    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:57.347113 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.347086    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-kmjpt","openshift-network-operator/iptables-alerter-z9qqt","openshift-ovn-kubernetes/ovnkube-node-9qd4q","kube-system/konnectivity-agent-5sv88","kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h","openshift-cluster-node-tuning-operator/tuned-lntbk","openshift-dns/node-resolver-rb5ff","openshift-image-registry/node-ca-dqwl4","openshift-multus/multus-additional-cni-plugins-kf66q","openshift-multus/multus-xqrvw","kube-system/global-pull-secret-syncer-r6dj8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal","openshift-multus/network-metrics-daemon-7ckzh"]
Jul 17 09:26:57.348407 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.348392    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 09:26:57.348471 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.348368    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 09:21:57 +0000 UTC" deadline="2028-04-19 18:31:46.52239694 +0000 UTC"
Jul 17 09:26:57.348471 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.348418    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15417h4m49.173982055s"
Jul 17 09:26:57.348999 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.348981    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.350984 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.350961    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.351099 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.351076    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.351309 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.351293    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f88sd\""
Jul 17 09:26:57.352094 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.352074    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.353668 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.353648    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.354026 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.353987    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.354122 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.354050    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.354213 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.354193    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qs7jz\""
Jul 17 09:26:57.354971 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.354955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysctl-conf\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355022 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.354983    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-systemd\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355022 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355008    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-sys\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355086 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e70b2c-a732-4ba8-b436-3caa5b43ccca-tmp-dir\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.355086 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355047    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysctl-d\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355086 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355050    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.355202 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355095    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxhvg\" (UniqueName: \"kubernetes.io/projected/9cedd655-45d3-4018-9ecb-b84169f91138-kube-api-access-xxhvg\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355202 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355143    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ttsl\" (UniqueName: \"kubernetes.io/projected/01e70b2c-a732-4ba8-b436-3caa5b43ccca-kube-api-access-7ttsl\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.355202 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355190    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-modprobe-d\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355345 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355233    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-kubernetes\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355345 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355260    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cedd655-45d3-4018-9ecb-b84169f91138-etc-tuned\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355345 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355275    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cedd655-45d3-4018-9ecb-b84169f91138-tmp\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355345 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355303    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysconfig\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355345 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355327    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-run\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355348    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-lib-modules\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-var-lib-kubelet\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355410    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-host\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.355528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.355442    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01e70b2c-a732-4ba8-b436-3caa5b43ccca-hosts-file\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.356175 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.356155    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.356383 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.356366    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:26:57.356464 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.356441    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:26:57.356658 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.356640    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 09:26:57.356658 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.356652    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hsgb5\""
Jul 17 09:26:57.356785 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.356730    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.356844 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.356808    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 09:26:57.357261 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.357235    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 09:26:57.357479 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.357461    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 09:26:57.357562 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.357532    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 09:26:57.357698 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.357680    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tz7hj\""
Jul 17 09:26:57.357759 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.357701    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 09:26:57.357811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.357766    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.359172 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.359151    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.359860 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.359843    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-48ck4\""
Jul 17 09:26:57.359967 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.359872    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.359967 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.359873    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.360083 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.360012    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 09:26:57.360472 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.360455    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:26:57.360877 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.360860    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:57.361037 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.360921    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:26:57.361213 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.361199    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.361649 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.361635    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.361832 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.361818    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 09:26:57.362280 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.362263    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.362642 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.362624    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s4rjt\""
Jul 17 09:26:57.363913 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.363885    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.364533 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.364510    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.364634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.364516    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.364700 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.364633    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pmkkw\""
Jul 17 09:26:57.364700 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.364673    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 09:26:57.365793 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.365771    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.366577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.366264    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 09:26:57.366577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.366372    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 09:26:57.366577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.366415    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zbtpr\""
Jul 17 09:26:57.366577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.366434    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 09:26:57.366806 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.366677    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 09:26:57.367724 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.367161    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 09:26:57.370055 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.368331    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 09:26:57.370055 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.368347    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6tqtg\""
Jul 17 09:26:57.370702 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.370686    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.370782 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.370760    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:26:57.389003 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.388981    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rxzs7"
Jul 17 09:26:57.396547 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.396522    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rxzs7"
Jul 17 09:26:57.450808 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.450779    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 09:26:57.456003 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.455961    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-system-cni-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456003 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-netns\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456003 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456025    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-conf-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00555f83-4bb4-4a88-9bb2-4b427ba80497-kubelet-config\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456084    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01e70b2c-a732-4ba8-b436-3caa5b43ccca-hosts-file\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01e70b2c-a732-4ba8-b436-3caa5b43ccca-hosts-file\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-socket-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456227    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bee86335-7cc3-42eb-ace0-7e456caa8a18-iptables-alerter-script\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456251    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-cni-multus\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456269    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-multus-certs\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456284    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00555f83-4bb4-4a88-9bb2-4b427ba80497-dbus\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456301    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-cni-netd\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456326    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-system-cni-dir\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-hostroot\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-daemon-config\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxhvg\" (UniqueName: \"kubernetes.io/projected/9cedd655-45d3-4018-9ecb-b84169f91138-kube-api-access-xxhvg\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456418    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf8dn\" (UniqueName: \"kubernetes.io/projected/35ca4b27-a6bf-445d-8525-9b93ec8ac565-kube-api-access-zf8dn\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456436    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-etc-kubernetes\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456459    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-modprobe-d\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.456634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456608    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cedd655-45d3-4018-9ecb-b84169f91138-tmp\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456675    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-modprobe-d\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456693    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-run-netns\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456737    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456784    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-k8s-cni-cncf-io\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456812    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-run\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-host\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cnibin\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456894    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-cni-bin\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456916    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-env-overrides\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-ovnkube-script-lib\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456949    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.456977    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wgqv\" (UniqueName: \"kubernetes.io/projected/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-kube-api-access-7wgqv\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457011    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-run\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457025    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-socket-dir-parent\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.457082 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457065    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-kubelet\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457094    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-cni-bin\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457139    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-sys-fs\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457152    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-host\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457163    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-cni-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457188    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-kubelet\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-var-lib-kubelet\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457274    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-var-lib-kubelet\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457303    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-etc-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457357    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457381    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457409    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-systemd-units\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457433    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f118c115-07a9-4529-ad62-7d077cec7cbe-ovn-node-metrics-cert\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457457    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bee86335-7cc3-42eb-ace0-7e456caa8a18-host-slash\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457478    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cni-binary-copy\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.457846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457493    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-cnibin\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-os-release\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457529    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4vc6\" (UniqueName: \"kubernetes.io/projected/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-kube-api-access-r4vc6\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457552    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-ovnkube-config\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457606    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjp99\" (UniqueName: \"kubernetes.io/projected/f118c115-07a9-4529-ad62-7d077cec7cbe-kube-api-access-sjp99\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457630    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-device-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457649    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-kubernetes\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457692    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cedd655-45d3-4018-9ecb-b84169f91138-etc-tuned\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457714    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-os-release\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457742    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-kubernetes\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457736    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-cni-binary-copy\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysconfig\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457843    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysconfig\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-lib-modules\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457894    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-slash\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457922    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-node-log\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.458735 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457945    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-log-socket\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457970    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cd669c69-0076-4509-b99c-40b3a747700f-agent-certs\") pod \"konnectivity-agent-5sv88\" (UID: \"cd669c69-0076-4509-b99c-40b3a747700f\") " pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.457994    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458020    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysctl-conf\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458047    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-systemd\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-sys\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e70b2c-a732-4ba8-b436-3caa5b43ccca-tmp-dir\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458124    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-var-lib-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458166    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffwg6\" (UniqueName: \"kubernetes.io/projected/82773457-e021-4aa2-84e9-7d1fe8045d0c-kube-api-access-ffwg6\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458193    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458218    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82773457-e021-4aa2-84e9-7d1fe8045d0c-host\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458232    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-lib-modules\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458243    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysctl-d\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458259    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-sys\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ttsl\" (UniqueName: \"kubernetes.io/projected/01e70b2c-a732-4ba8-b436-3caa5b43ccca-kube-api-access-7ttsl\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458375    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-systemd\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458385    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysctl-d\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.459440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458427    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-systemd\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458556    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cd669c69-0076-4509-b99c-40b3a747700f-konnectivity-ca\") pod \"konnectivity-agent-5sv88\" (UID: \"cd669c69-0076-4509-b99c-40b3a747700f\") " pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458591    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-registration-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458633    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e70b2c-a732-4ba8-b436-3caa5b43ccca-tmp-dir\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbc7j\" (UniqueName: \"kubernetes.io/projected/b71746b5-6537-4597-96d0-adc05704a69e-kube-api-access-fbc7j\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458679    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/82773457-e021-4aa2-84e9-7d1fe8045d0c-serviceca\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458707    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458710    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cedd655-45d3-4018-9ecb-b84169f91138-etc-sysctl-conf\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458750    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-ovn\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458783    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69fmh\" (UniqueName: \"kubernetes.io/projected/bee86335-7cc3-42eb-ace0-7e456caa8a18-kube-api-access-69fmh\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.460123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.458811    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.460534 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.460458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cedd655-45d3-4018-9ecb-b84169f91138-tmp\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.460572 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.460537    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cedd655-45d3-4018-9ecb-b84169f91138-etc-tuned\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.464527 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.464502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxhvg\" (UniqueName: \"kubernetes.io/projected/9cedd655-45d3-4018-9ecb-b84169f91138-kube-api-access-xxhvg\") pod \"tuned-lntbk\" (UID: \"9cedd655-45d3-4018-9ecb-b84169f91138\") " pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.467945 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.467912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ttsl\" (UniqueName: \"kubernetes.io/projected/01e70b2c-a732-4ba8-b436-3caa5b43ccca-kube-api-access-7ttsl\") pod \"node-resolver-rb5ff\" (UID: \"01e70b2c-a732-4ba8-b436-3caa5b43ccca\") " pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.559118 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559078    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.559118 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-ovn\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559149    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69fmh\" (UniqueName: \"kubernetes.io/projected/bee86335-7cc3-42eb-ace0-7e456caa8a18-kube-api-access-69fmh\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559174    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559195    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-system-cni-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559200    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-ovn\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559217    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-netns\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559241    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-conf-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559264    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00555f83-4bb4-4a88-9bb2-4b427ba80497-kubelet-config\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559286    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559313    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-socket-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559327    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bee86335-7cc3-42eb-ace0-7e456caa8a18-iptables-alerter-script\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559330    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-netns\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559365    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-cni-multus\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559367    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/00555f83-4bb4-4a88-9bb2-4b427ba80497-kubelet-config\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.559401 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559368    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-conf-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559310    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-system-cni-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559429    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-cni-multus\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559442    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-socket-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.559446    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559471    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-multus-certs\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559504    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00555f83-4bb4-4a88-9bb2-4b427ba80497-dbus\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559535    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-cni-netd\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559535    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-multus-certs\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.559558    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:58.059534298 +0000 UTC m=+2.069658225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559569    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-cni-netd\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559596    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-system-cni-dir\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-hostroot\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559644    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-daemon-config\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/00555f83-4bb4-4a88-9bb2-4b427ba80497-dbus\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-system-cni-dir\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-hostroot\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.560218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559727    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf8dn\" (UniqueName: \"kubernetes.io/projected/35ca4b27-a6bf-445d-8525-9b93ec8ac565-kube-api-access-zf8dn\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559749    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-etc-kubernetes\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559768    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559796    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-run-netns\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559816    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559850    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-k8s-cni-cncf-io\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559857    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-etc-kubernetes\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559874    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-run-netns\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-run-k8s-cni-cncf-io\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559897    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cnibin\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559905    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-cni-bin\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559954    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cnibin\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-cni-bin\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559963    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-env-overrides\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.561145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.559990    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-ovnkube-script-lib\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560017    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wgqv\" (UniqueName: \"kubernetes.io/projected/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-kube-api-access-7wgqv\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560035    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-socket-dir-parent\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-kubelet\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560082    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-cni-bin\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560099    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-sys-fs\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-cni-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560131    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-socket-dir-parent\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-kubelet\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560177    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-etc-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560190    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-cni-bin\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560208    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560234    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-kubelet\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560275    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-host-var-lib-kubelet\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560273    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560295    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-sys-fs\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.562001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560315    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-systemd-units\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f118c115-07a9-4529-ad62-7d077cec7cbe-ovn-node-metrics-cert\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560360    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-cni-dir\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560372    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bee86335-7cc3-42eb-ace0-7e456caa8a18-host-slash\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560399    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cni-binary-copy\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560425    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-cnibin\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560454    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-os-release\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560482    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4vc6\" (UniqueName: \"kubernetes.io/projected/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-kube-api-access-r4vc6\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560516    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-ovnkube-config\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560516    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-env-overrides\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjp99\" (UniqueName: \"kubernetes.io/projected/f118c115-07a9-4529-ad62-7d077cec7cbe-kube-api-access-sjp99\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-device-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560576    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560602    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560610    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bee86335-7cc3-42eb-ace0-7e456caa8a18-iptables-alerter-script\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560624    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-etc-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560641    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-os-release\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.562740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560400    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-systemd-units\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560689    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-cni-binary-copy\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560699    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-cnibin\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560245    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-multus-daemon-config\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560740    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-slash\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560745    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bee86335-7cc3-42eb-ace0-7e456caa8a18-host-slash\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560751    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-node-log\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560777    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-ovnkube-script-lib\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560795    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-log-socket\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560811    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-os-release\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560833    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-log-socket\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560835    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cd669c69-0076-4509-b99c-40b3a747700f-agent-certs\") pod \"konnectivity-agent-5sv88\" (UID: \"cd669c69-0076-4509-b99c-40b3a747700f\") " pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560850    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-host-slash\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560867    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-var-lib-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563285 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.560974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffwg6\" (UniqueName: \"kubernetes.io/projected/82773457-e021-4aa2-84e9-7d1fe8045d0c-kube-api-access-ffwg6\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561002    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561029    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82773457-e021-4aa2-84e9-7d1fe8045d0c-host\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561056    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-systemd\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561080    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cd669c69-0076-4509-b99c-40b3a747700f-konnectivity-ca\") pod \"konnectivity-agent-5sv88\" (UID: \"cd669c69-0076-4509-b99c-40b3a747700f\") " pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561172    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-device-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561246    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35ca4b27-a6bf-445d-8525-9b93ec8ac565-os-release\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561273    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-cni-binary-copy\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561328    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-node-log\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561352    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82773457-e021-4aa2-84e9-7d1fe8045d0c-host\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561383    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-var-lib-openvswitch\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561446    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f118c115-07a9-4529-ad62-7d077cec7cbe-run-systemd\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-registration-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561542    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbc7j\" (UniqueName: \"kubernetes.io/projected/b71746b5-6537-4597-96d0-adc05704a69e-kube-api-access-fbc7j\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561569    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/82773457-e021-4aa2-84e9-7d1fe8045d0c-serviceca\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561599    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cd669c69-0076-4509-b99c-40b3a747700f-konnectivity-ca\") pod \"konnectivity-agent-5sv88\" (UID: \"cd669c69-0076-4509-b99c-40b3a747700f\") " pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.561604    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b71746b5-6537-4597-96d0-adc05704a69e-registration-dir\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.563885 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.561658    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:57.564544 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.561729    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:26:58.061707873 +0000 UTC m=+2.071831779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:57.564544 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.562023    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f118c115-07a9-4529-ad62-7d077cec7cbe-ovnkube-config\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.564544 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.562061    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35ca4b27-a6bf-445d-8525-9b93ec8ac565-cni-binary-copy\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.564544 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.562427    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/82773457-e021-4aa2-84e9-7d1fe8045d0c-serviceca\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.564544 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.563133    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f118c115-07a9-4529-ad62-7d077cec7cbe-ovn-node-metrics-cert\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.564544 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.563736    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cd669c69-0076-4509-b99c-40b3a747700f-agent-certs\") pod \"konnectivity-agent-5sv88\" (UID: \"cd669c69-0076-4509-b99c-40b3a747700f\") " pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.569295 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.569237    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:57.569295 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.569270    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:57.569581 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.569329    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69fmh\" (UniqueName: \"kubernetes.io/projected/bee86335-7cc3-42eb-ace0-7e456caa8a18-kube-api-access-69fmh\") pod \"iptables-alerter-z9qqt\" (UID: \"bee86335-7cc3-42eb-ace0-7e456caa8a18\") " pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.569581 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.569287    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:57.569581 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:57.569435    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:26:58.069411849 +0000 UTC m=+2.079535771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:57.569581 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.569519    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf8dn\" (UniqueName: \"kubernetes.io/projected/35ca4b27-a6bf-445d-8525-9b93ec8ac565-kube-api-access-zf8dn\") pod \"multus-additional-cni-plugins-kf66q\" (UID: \"35ca4b27-a6bf-445d-8525-9b93ec8ac565\") " pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.571615 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.571586    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbc7j\" (UniqueName: \"kubernetes.io/projected/b71746b5-6537-4597-96d0-adc05704a69e-kube-api-access-fbc7j\") pod \"aws-ebs-csi-driver-node-qlv2h\" (UID: \"b71746b5-6537-4597-96d0-adc05704a69e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.572243 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.572216    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4vc6\" (UniqueName: \"kubernetes.io/projected/b3ebbbf6-5a57-4076-9df0-378c9aaefb78-kube-api-access-r4vc6\") pod \"multus-xqrvw\" (UID: \"b3ebbbf6-5a57-4076-9df0-378c9aaefb78\") " pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.572421 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.572396    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjp99\" (UniqueName: \"kubernetes.io/projected/f118c115-07a9-4529-ad62-7d077cec7cbe-kube-api-access-sjp99\") pod \"ovnkube-node-9qd4q\" (UID: \"f118c115-07a9-4529-ad62-7d077cec7cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.572617 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.572588    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffwg6\" (UniqueName: \"kubernetes.io/projected/82773457-e021-4aa2-84e9-7d1fe8045d0c-kube-api-access-ffwg6\") pod \"node-ca-dqwl4\" (UID: \"82773457-e021-4aa2-84e9-7d1fe8045d0c\") " pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.573118 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.573097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wgqv\" (UniqueName: \"kubernetes.io/projected/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-kube-api-access-7wgqv\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:57.588446 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.588411    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d37f3cbe072cd8ae509ba808ddfa814.slice/crio-01c911df5120350f70beb6a5c2635f7005338110ed4a49bd337f64372fa7dfd3 WatchSource:0}: Error finding container 01c911df5120350f70beb6a5c2635f7005338110ed4a49bd337f64372fa7dfd3: Status 404 returned error can't find the container with id 01c911df5120350f70beb6a5c2635f7005338110ed4a49bd337f64372fa7dfd3
Jul 17 09:26:57.589106 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.589075    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3895fcc55e8580e43648b59a9ffbae.slice/crio-5a01351df7b7925a2f73db2b710a83bc614bbb3a1782ecebb4139f9daf44d957 WatchSource:0}: Error finding container 5a01351df7b7925a2f73db2b710a83bc614bbb3a1782ecebb4139f9daf44d957: Status 404 returned error can't find the container with id 5a01351df7b7925a2f73db2b710a83bc614bbb3a1782ecebb4139f9daf44d957
Jul 17 09:26:57.594180 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.594160    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:26:57.681041 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.680997    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lntbk"
Jul 17 09:26:57.687371 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.687342    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rb5ff"
Jul 17 09:26:57.690668 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.690638    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cedd655_45d3_4018_9ecb_b84169f91138.slice/crio-d53d95045bb342d09b00066f38a4591175b65e5927fa84916f86e1d7da2ca7f4 WatchSource:0}: Error finding container d53d95045bb342d09b00066f38a4591175b65e5927fa84916f86e1d7da2ca7f4: Status 404 returned error can't find the container with id d53d95045bb342d09b00066f38a4591175b65e5927fa84916f86e1d7da2ca7f4
Jul 17 09:26:57.696965 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.696934    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e70b2c_a732_4ba8_b436_3caa5b43ccca.slice/crio-7ea9515f6793047d273478e984a16ae90cb96eac44539fa1ba36d6345678dc9a WatchSource:0}: Error finding container 7ea9515f6793047d273478e984a16ae90cb96eac44539fa1ba36d6345678dc9a: Status 404 returned error can't find the container with id 7ea9515f6793047d273478e984a16ae90cb96eac44539fa1ba36d6345678dc9a
Jul 17 09:26:57.714586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.714553    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:26:57.726461 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.726417    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:26:57.735086 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.735054    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd669c69_0076_4509_b99c_40b3a747700f.slice/crio-571e80eda1153fd057cf7b7c892c52cbcd61c6ce58256a04451b86a9ada473f6 WatchSource:0}: Error finding container 571e80eda1153fd057cf7b7c892c52cbcd61c6ce58256a04451b86a9ada473f6: Status 404 returned error can't find the container with id 571e80eda1153fd057cf7b7c892c52cbcd61c6ce58256a04451b86a9ada473f6
Jul 17 09:26:57.741249 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.741223    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h"
Jul 17 09:26:57.749625 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.749593    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71746b5_6537_4597_96d0_adc05704a69e.slice/crio-4b1016e600d43853124fd374d0d52ffe81c591e3a6a2330119f983d8bca6b4aa WatchSource:0}: Error finding container 4b1016e600d43853124fd374d0d52ffe81c591e3a6a2330119f983d8bca6b4aa: Status 404 returned error can't find the container with id 4b1016e600d43853124fd374d0d52ffe81c591e3a6a2330119f983d8bca6b4aa
Jul 17 09:26:57.760681 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.760647    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dqwl4"
Jul 17 09:26:57.769324 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.769289    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82773457_e021_4aa2_84e9_7d1fe8045d0c.slice/crio-cf2fa20b545c14071cd67d8ac16abaae2b958aa664966af07798a08047d1a7ea WatchSource:0}: Error finding container cf2fa20b545c14071cd67d8ac16abaae2b958aa664966af07798a08047d1a7ea: Status 404 returned error can't find the container with id cf2fa20b545c14071cd67d8ac16abaae2b958aa664966af07798a08047d1a7ea
Jul 17 09:26:57.776999 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.776976    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9qqt"
Jul 17 09:26:57.785541 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.785504    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee86335_7cc3_42eb_ace0_7e456caa8a18.slice/crio-8b32e86df9144bb44147ff5457825524f696f49ab6c3f226b3ffbc4a01646fa2 WatchSource:0}: Error finding container 8b32e86df9144bb44147ff5457825524f696f49ab6c3f226b3ffbc4a01646fa2: Status 404 returned error can't find the container with id 8b32e86df9144bb44147ff5457825524f696f49ab6c3f226b3ffbc4a01646fa2
Jul 17 09:26:57.798648 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.798616    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kf66q"
Jul 17 09:26:57.805140 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:57.805104    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xqrvw"
Jul 17 09:26:57.807887 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.807858    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ca4b27_a6bf_445d_8525_9b93ec8ac565.slice/crio-bf5d15842533fcbba6968e6c595bc256f2f531b5fc0180fab890c0a27fc6e231 WatchSource:0}: Error finding container bf5d15842533fcbba6968e6c595bc256f2f531b5fc0180fab890c0a27fc6e231: Status 404 returned error can't find the container with id bf5d15842533fcbba6968e6c595bc256f2f531b5fc0180fab890c0a27fc6e231
Jul 17 09:26:57.814562 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:26:57.814525    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ebbbf6_5a57_4076_9df0_378c9aaefb78.slice/crio-4d8a02a7d7e422512a658a5275d8d843674f9915ecefc2e2cf700f0c6067ac7e WatchSource:0}: Error finding container 4d8a02a7d7e422512a658a5275d8d843674f9915ecefc2e2cf700f0c6067ac7e: Status 404 returned error can't find the container with id 4d8a02a7d7e422512a658a5275d8d843674f9915ecefc2e2cf700f0c6067ac7e
Jul 17 09:26:58.065141 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.065036    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:58.065141 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.065101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:58.065380 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.065211    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:58.065380 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.065213    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:58.065380 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.065271    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:26:59.065255379 +0000 UTC m=+3.075379283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:58.065380 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.065292    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:59.065280744 +0000 UTC m=+3.075404664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:58.166364 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.166329    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:26:58.166559 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.166545    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:58.166622 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.166572    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:58.166622 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.166583    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:58.166752 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:58.166647    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:26:59.166628222 +0000 UTC m=+3.176752126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:58.323972 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.323870    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:58.399380 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.399275    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:21:58 +0000 UTC" deadline="2027-12-19 13:12:44.130902699 +0000 UTC"
Jul 17 09:26:58.399380 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.399351    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12483h45m45.731556781s"
Jul 17 09:26:58.515443 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.515363    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h" event={"ID":"b71746b5-6537-4597-96d0-adc05704a69e","Type":"ContainerStarted","Data":"4b1016e600d43853124fd374d0d52ffe81c591e3a6a2330119f983d8bca6b4aa"}
Jul 17 09:26:58.528010 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.527582    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rb5ff" event={"ID":"01e70b2c-a732-4ba8-b436-3caa5b43ccca","Type":"ContainerStarted","Data":"7ea9515f6793047d273478e984a16ae90cb96eac44539fa1ba36d6345678dc9a"}
Jul 17 09:26:58.535748 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.535697    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lntbk" event={"ID":"9cedd655-45d3-4018-9ecb-b84169f91138","Type":"ContainerStarted","Data":"d53d95045bb342d09b00066f38a4591175b65e5927fa84916f86e1d7da2ca7f4"}
Jul 17 09:26:58.550932 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.550861    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal" event={"ID":"2b3895fcc55e8580e43648b59a9ffbae","Type":"ContainerStarted","Data":"5a01351df7b7925a2f73db2b710a83bc614bbb3a1782ecebb4139f9daf44d957"}
Jul 17 09:26:58.569483 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.569418    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerStarted","Data":"bf5d15842533fcbba6968e6c595bc256f2f531b5fc0180fab890c0a27fc6e231"}
Jul 17 09:26:58.575116 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.575020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9qqt" event={"ID":"bee86335-7cc3-42eb-ace0-7e456caa8a18","Type":"ContainerStarted","Data":"8b32e86df9144bb44147ff5457825524f696f49ab6c3f226b3ffbc4a01646fa2"}
Jul 17 09:26:58.582691 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.582642    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dqwl4" event={"ID":"82773457-e021-4aa2-84e9-7d1fe8045d0c","Type":"ContainerStarted","Data":"cf2fa20b545c14071cd67d8ac16abaae2b958aa664966af07798a08047d1a7ea"}
Jul 17 09:26:58.598127 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.598081    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5sv88" event={"ID":"cd669c69-0076-4509-b99c-40b3a747700f","Type":"ContainerStarted","Data":"571e80eda1153fd057cf7b7c892c52cbcd61c6ce58256a04451b86a9ada473f6"}
Jul 17 09:26:58.628326 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.628293    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:58.628539 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.628297    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"0f9cdb59722e768c17d8cd906fbd6496736eb23506c2b7b02493b5b50ec98f6a"}
Jul 17 09:26:58.650140 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.650098    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal" event={"ID":"0d37f3cbe072cd8ae509ba808ddfa814","Type":"ContainerStarted","Data":"01c911df5120350f70beb6a5c2635f7005338110ed4a49bd337f64372fa7dfd3"}
Jul 17 09:26:58.668159 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:58.668066    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xqrvw" event={"ID":"b3ebbbf6-5a57-4076-9df0-378c9aaefb78","Type":"ContainerStarted","Data":"4d8a02a7d7e422512a658a5275d8d843674f9915ecefc2e2cf700f0c6067ac7e"}
Jul 17 09:26:59.075791 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.074781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:59.075791 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.074887    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:59.075791 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.075047    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:59.075791 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.075118    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:01.075098562 +0000 UTC m=+5.085222481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:59.075791 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.075626    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:59.075791 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.075686    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:01.075666849 +0000 UTC m=+5.085790764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:59.176526 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.175727    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:26:59.176526 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.175989    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:59.176526 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.176012    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:59.176526 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.176025    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:59.176526 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.176086    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:01.176067333 +0000 UTC m=+5.186191237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:59.404886 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.399627    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:21:58 +0000 UTC" deadline="2028-04-06 07:10:53.880473316 +0000 UTC"
Jul 17 09:26:59.404886 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.399832    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15093h43m54.48064649s"
Jul 17 09:26:59.490160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.490123    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:26:59.490358 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.490272    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:26:59.490750 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.490727    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:26:59.490869 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.490832    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:26:59.490946 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.490936    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:26:59.491058 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:26:59.491035    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:26:59.804754 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:26:59.804661    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:27:01.094412 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:01.094367    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:01.094941 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:01.094472    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:01.094941 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.094607    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:01.094941 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.094680    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:05.094658993 +0000 UTC m=+9.104782957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:01.095268 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.095234    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:01.095360 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.095290    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:05.095277265 +0000 UTC m=+9.105401169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:01.195697 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:01.195560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:01.195921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.195757    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:27:01.195921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.195781    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:27:01.195921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.195794    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:01.195921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.195875    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:05.195852397 +0000 UTC m=+9.205976316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:01.489982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:01.489246    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:01.489982 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.489377    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:01.489982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:01.489685    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:01.489982 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.489782    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:01.489982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:01.489848    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:01.490363 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:01.490011    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:03.489649 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:03.489607    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:03.489649 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:03.489656    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:03.490284 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:03.489621    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:03.490284 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:03.489754    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:03.490284 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:03.489910    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:03.490284 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:03.489993    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:05.132241 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:05.132197    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:05.132726 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:05.132293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:05.132726 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.132382    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:05.132726 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.132436    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:05.132726 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.132469    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:13.132440502 +0000 UTC m=+17.142564431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:05.132726 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.132508    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:13.132488952 +0000 UTC m=+17.142612858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:05.232663 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:05.232621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:05.232921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.232794    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:27:05.232921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.232820    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:27:05.232921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.232834    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:05.232921 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.232917    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:13.232899806 +0000 UTC m=+17.243023709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:05.490391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:05.490292    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:05.490551 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.490432    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:05.490794 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:05.490778    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:05.490876 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.490860    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:05.490959 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:05.490945    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:05.491131 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:05.491113    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:07.489939 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:07.489894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:07.490447 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:07.489948    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:07.490447 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:07.490039    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:07.490447 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:07.490115    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:07.490447 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:07.490294    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:07.490447 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:07.490430    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:09.489915 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:09.489880    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:09.490316 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:09.489880    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:09.490316 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:09.490009    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:09.490316 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:09.489881    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:09.490316 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:09.490082    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:09.490316 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:09.490159    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:11.490184 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:11.490104    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:11.490645 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:11.490104    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:11.490645 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:11.490233    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:11.490645 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:11.490332    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:11.490645 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:11.490103    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:11.490645 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:11.490428    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:13.201056 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:13.201007    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:13.201445 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:13.201092    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:13.201445 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.201164    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:13.201445 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.201181    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:13.201445 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.201236    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:29.201215693 +0000 UTC m=+33.211339594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:13.201445 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.201250    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:29.20124399 +0000 UTC m=+33.211367891 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:13.301888 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:13.301834    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:13.302080 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.301976    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:27:13.302080 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.302001    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:27:13.302080 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.302018    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:13.302232 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.302091    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:29.302075604 +0000 UTC m=+33.312199506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:13.490032 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:13.489951    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:13.490183 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:13.489957    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:13.490183 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.490081    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:13.490183 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:13.489965    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:13.490318 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.490136    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:13.490318 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:13.490282    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:15.490228 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:15.490186    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:15.490719 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:15.490243    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:15.490719 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:15.490330    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:15.490719 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:15.490346    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:15.490719 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:15.490444    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:15.490719 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:15.490529    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:17.489768 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:17.489723    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:17.490235 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:17.489723    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:17.490235 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:17.489863    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:17.490235 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:17.489952    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:17.490235 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:17.489723    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:17.490235 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:17.490072    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:19.489943 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.489344    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:19.489943 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:19.489653    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:19.489943 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.489385    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:19.489943 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:19.489750    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:19.489943 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.489344    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:19.489943 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:19.489868    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:19.721226 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.721189    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lntbk" event={"ID":"9cedd655-45d3-4018-9ecb-b84169f91138","Type":"ContainerStarted","Data":"b5f79bd66a8aeffa2110fee41c3150cdbe930aef913953bab4c788b9cd1a79ec"}
Jul 17 09:27:19.722942 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.722879    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal" event={"ID":"2b3895fcc55e8580e43648b59a9ffbae","Type":"ContainerStarted","Data":"f8c5b82985311fb7ec626ad276f3f0efd09436bcfc6a447a2af751a8d7715a4d"}
Jul 17 09:27:19.725275 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.725236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"9373eb842f80bf2a5eada88b8a8eb8b24f7e746ad05b620fa14c4cbd1f8d1fe5"}
Jul 17 09:27:19.725388 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.725315    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"fd84c6bdc3a7aa482a8223e07eb090ba7e96310664467845e31935ec374e2531"}
Jul 17 09:27:19.727172 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.727045    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xqrvw" event={"ID":"b3ebbbf6-5a57-4076-9df0-378c9aaefb78","Type":"ContainerStarted","Data":"6b81a516a3fdd30c2c56ad02ffed579e79495d1702dba08fa37106bedc794785"}
Jul 17 09:27:19.758427 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.758371    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xqrvw" podStartSLOduration=1.366180767 podStartE2EDuration="22.758351615s" totalImagesPullingTime="21.392170848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:19.758316661 +0000 UTC m=+23.768440586" watchObservedRunningTime="2026-07-17 09:27:19.758351615 +0000 UTC m=+23.768475537"
Jul 17 09:27:19.758956 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.758915    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lntbk" podStartSLOduration=1.272278461 podStartE2EDuration="22.75890463s" totalImagesPullingTime="21.486626169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:19.742058979 +0000 UTC m=+23.752182904" watchObservedRunningTime="2026-07-17 09:27:19.75890463 +0000 UTC m=+23.769028553"
Jul 17 09:27:19.770059 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:19.770014    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-244.ec2.internal" podStartSLOduration=21.769998036 podStartE2EDuration="21.769998036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:19.769633405 +0000 UTC m=+23.779757328" watchObservedRunningTime="2026-07-17 09:27:19.769998036 +0000 UTC m=+23.780121959"
Jul 17 09:27:20.731109 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.730807    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ca4b27-a6bf-445d-8525-9b93ec8ac565" containerID="b33fad8dabc01901867d94be07ae6ca1d2dd9703f9fe97558d8dca4b7c4d9720" exitCode=0
Jul 17 09:27:20.731943 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.730871    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerDied","Data":"b33fad8dabc01901867d94be07ae6ca1d2dd9703f9fe97558d8dca4b7c4d9720"}
Jul 17 09:27:20.732781 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.732757    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dqwl4" event={"ID":"82773457-e021-4aa2-84e9-7d1fe8045d0c","Type":"ContainerStarted","Data":"5d9fcad75eeee49c7e23948f352a2b147e843ec0b990da90a34018e41a46a78f"}
Jul 17 09:27:20.734484 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.734451    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5sv88" event={"ID":"cd669c69-0076-4509-b99c-40b3a747700f","Type":"ContainerStarted","Data":"4a109043e5e209fdb80fc883d8d64fc52d3925a6b1eded02a90084d6fd4a8747"}
Jul 17 09:27:20.737838 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.737783    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"14d407ca8e8dcd46e5fe2e283ea817b12c780b9ae02d1969763df8d02603e69d"}
Jul 17 09:27:20.737987 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.737899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"67502875dfb51d4c7bc1a6a4f742e89e7deaa073dc052ebf59b8ee2f631b687f"}
Jul 17 09:27:20.737987 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.737949    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"9e6ba93a52471d305313b043ddf5124f8269f977cd3c1a6d589a9f70d46a7520"}
Jul 17 09:27:20.738099 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.737998    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"c4a50f7379756876bfd090d8f315e3a1a9d0e1efee85fc40606f54d6821d008c"}
Jul 17 09:27:20.739453 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.739429    2556 generic.go:356] "Generic (PLEG): container finished" podID="0d37f3cbe072cd8ae509ba808ddfa814" containerID="e68c9fafde798f9227db316b48cdd5cf96e2c974a91ea4bba06f059df0cfd5d7" exitCode=0
Jul 17 09:27:20.739586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.739502    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal" event={"ID":"0d37f3cbe072cd8ae509ba808ddfa814","Type":"ContainerDied","Data":"e68c9fafde798f9227db316b48cdd5cf96e2c974a91ea4bba06f059df0cfd5d7"}
Jul 17 09:27:20.740956 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.740935    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h" event={"ID":"b71746b5-6537-4597-96d0-adc05704a69e","Type":"ContainerStarted","Data":"4ed8474c531d4d87cf4c2ae39fdf3118e1a4aecff6deb1c19a2702a0967bb125"}
Jul 17 09:27:20.744899 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.744834    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rb5ff" event={"ID":"01e70b2c-a732-4ba8-b436-3caa5b43ccca","Type":"ContainerStarted","Data":"bebe93e90055a13cf64af8d651a369934bd81359be01065e3eb0515c06dd7880"}
Jul 17 09:27:20.776917 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.776862    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5sv88" podStartSLOduration=2.348850981 podStartE2EDuration="23.776843519s" totalImagesPullingTime="21.427992538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:20.776548564 +0000 UTC m=+24.786672487" watchObservedRunningTime="2026-07-17 09:27:20.776843519 +0000 UTC m=+24.786967439"
Jul 17 09:27:20.777184 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.777130    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rb5ff" podStartSLOduration=2.307269977 podStartE2EDuration="23.777118123s" totalImagesPullingTime="21.469848146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:20.762157177 +0000 UTC m=+24.772281104" watchObservedRunningTime="2026-07-17 09:27:20.777118123 +0000 UTC m=+24.787242048"
Jul 17 09:27:20.790237 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.790184    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dqwl4" podStartSLOduration=2.396343461 podStartE2EDuration="23.790166926s" totalImagesPullingTime="21.393823465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:20.790004328 +0000 UTC m=+24.800128263" watchObservedRunningTime="2026-07-17 09:27:20.790166926 +0000 UTC m=+24.800290850"
Jul 17 09:27:20.876144 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.876103    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:20.950995 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:20.950955    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:21.340901 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.340864    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:21.379814 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.379784    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 09:27:21.415907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.415770    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T09:27:21.379805402Z","UUID":"39ad4bcb-b7c3-4db1-83d0-a0576c0c40e0","Handler":null,"Name":"","Endpoint":""}
Jul 17 09:27:21.417490 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.417471    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 09:27:21.417653 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.417514    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 09:27:21.441328 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.441298    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:21.489655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.489619    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:21.489655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.489648    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:21.489924 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.489672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:21.489924 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:21.489755    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:21.489924 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:21.489837    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:21.489924 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:21.489901    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:21.750069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.749994    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal" event={"ID":"0d37f3cbe072cd8ae509ba808ddfa814","Type":"ContainerStarted","Data":"e8f5d11746696e7091e2a60f6508835214a3ed666a0705151c04a8784b9f1809"}
Jul 17 09:27:21.752097 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.752067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h" event={"ID":"b71746b5-6537-4597-96d0-adc05704a69e","Type":"ContainerStarted","Data":"c2e0a8361a58200381761a8af340d20888449638ef271ac51fe78ee296c25cd2"}
Jul 17 09:27:21.753991 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.753930    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9qqt" event={"ID":"bee86335-7cc3-42eb-ace0-7e456caa8a18","Type":"ContainerStarted","Data":"d90615bdbeea4b85752c876f1b1c409827910fd94c6350c4390ceb72b518d4a3"}
Jul 17 09:27:21.764316 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.764247    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-244.ec2.internal" podStartSLOduration=23.764228966 podStartE2EDuration="23.764228966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:21.763719798 +0000 UTC m=+25.773843723" watchObservedRunningTime="2026-07-17 09:27:21.764228966 +0000 UTC m=+25.774352883"
Jul 17 09:27:21.781077 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.781016    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z9qqt" podStartSLOduration=3.403563114 podStartE2EDuration="24.780994184s" totalImagesPullingTime="21.37743107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:21.780874754 +0000 UTC m=+25.790998679" watchObservedRunningTime="2026-07-17 09:27:21.780994184 +0000 UTC m=+25.791118108"
Jul 17 09:27:21.847723 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.847689    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:27:21.848312 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:21.848287    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:27:22.031026 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:22.030932    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:22.354721 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:22.354637    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:22.455769 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:22.455733    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:22.555796 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:22.555756    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:22.762027 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:22.761993    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"21102167031e2ed9b6e0d8c6d26fda43ff99c7581950cf955949932c800dcda9"}
Jul 17 09:27:23.489272 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:23.489234    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:23.489272 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:23.489269    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:23.489497 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:23.489246    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:23.489497 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:23.489368    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:23.489497 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:23.489437    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:23.489595 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:23.489530    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:23.766972 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:23.766894    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 09:27:23.766972 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:23.766895    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h" event={"ID":"b71746b5-6537-4597-96d0-adc05704a69e","Type":"ContainerStarted","Data":"d4386e0435c1124f3e3c65993b8fd0248d64468ea7b481d21f024053751c5403"}
Jul 17 09:27:23.784450 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:23.784396    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlv2h" podStartSLOduration=2.764986149 podStartE2EDuration="26.784379057s" totalImagesPullingTime="24.019392908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:23.784163048 +0000 UTC m=+27.794286975" watchObservedRunningTime="2026-07-17 09:27:23.784379057 +0000 UTC m=+27.794502981"
Jul 17 09:27:24.232443 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:24.232236    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:24.335056 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:24.335020    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:25.489807 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.489553    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:25.490655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.489553    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:25.490655 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:25.489864    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:25.490655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.489553    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:25.490655 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:25.489920    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:25.490655 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:25.489993    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:25.638320 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.638279    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:27:25.638522 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.638423    2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 09:27:25.638914 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.638891    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5sv88"
Jul 17 09:27:25.700451 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.700418    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:25.774738 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.774641    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" event={"ID":"f118c115-07a9-4529-ad62-7d077cec7cbe","Type":"ContainerStarted","Data":"51317067efd24ff374e4f0338e74ac006cc68f4500e876c1a04fa40a6f8cde30"}
Jul 17 09:27:25.774988 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.774973    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:27:25.775086 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.775043    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:27:25.776583 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.776555    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ca4b27-a6bf-445d-8525-9b93ec8ac565" containerID="078a1317b85c50d2104fe98b3907e00cbb59c8a5c5017f5ac9b69d00753746b4" exitCode=0
Jul 17 09:27:25.776688 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.776632    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerDied","Data":"078a1317b85c50d2104fe98b3907e00cbb59c8a5c5017f5ac9b69d00753746b4"}
Jul 17 09:27:25.792455 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.792429    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:27:25.797545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:25.797486    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q" podStartSLOduration=7.292214802 podStartE2EDuration="28.797466333s" totalImagesPullingTime="21.505251531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:25.796658939 +0000 UTC m=+29.806782915" watchObservedRunningTime="2026-07-17 09:27:25.797466333 +0000 UTC m=+29.807590250"
Jul 17 09:27:26.779477 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.779445    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:27:26.796733 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.796704    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:27:26.948599 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.948570    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7ckzh"]
Jul 17 09:27:26.948752 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.948716    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:26.948878 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:26.948856    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:26.951361 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.951337    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kmjpt"]
Jul 17 09:27:26.951473 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.951453    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:26.951587 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:26.951563    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:26.961715 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.961686    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r6dj8"]
Jul 17 09:27:26.961894 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:26.961811    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:26.962019 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:26.961993    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:27.783918 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:27.783883    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ca4b27-a6bf-445d-8525-9b93ec8ac565" containerID="06ac4070405d64e12d8ce5baef346ee5b7747bb47334a47808e35c203a2b3876" exitCode=0
Jul 17 09:27:27.784293 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:27.783977    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerDied","Data":"06ac4070405d64e12d8ce5baef346ee5b7747bb47334a47808e35c203a2b3876"}
Jul 17 09:27:28.415152 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:28.415111    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:28.490185 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:28.490154    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:28.490351 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:28.490278    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:28.490351 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:28.490292    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:28.490463 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:28.490410    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:28.490504 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:28.490475    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:28.490570 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:28.490553    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:29.212263 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:29.212073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:29.212675 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:29.212314    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:29.212675 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.212211    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:29.212675 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.212392    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret podName:00555f83-4bb4-4a88-9bb2-4b427ba80497 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:01.212373046 +0000 UTC m=+65.222496952 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret") pod "global-pull-secret-syncer-r6dj8" (UID: "00555f83-4bb4-4a88-9bb2-4b427ba80497") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:29.212675 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.212447    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:29.212675 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.212500    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs podName:d9c0f5b2-fd40-4bc0-a326-4ca7a466044c nodeName:}" failed. No retries permitted until 2026-07-17 09:28:01.21248679 +0000 UTC m=+65.222610691 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs") pod "network-metrics-daemon-7ckzh" (UID: "d9c0f5b2-fd40-4bc0-a326-4ca7a466044c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:29.312754 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:29.312650    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:29.312926 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.312847    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:27:29.312926 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.312869    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:27:29.312926 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.312881    2556 projected.go:196] Error preparing data for projected volume kube-api-access-g4svj for pod openshift-network-diagnostics/network-check-target-kmjpt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:29.313027 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:29.312933    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj podName:5bffb080-9b11-4597-ba34-63f4ecfdf75e nodeName:}" failed. No retries permitted until 2026-07-17 09:28:01.312918106 +0000 UTC m=+65.323042033 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g4svj" (UniqueName: "kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj") pod "network-check-target-kmjpt" (UID: "5bffb080-9b11-4597-ba34-63f4ecfdf75e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:29.792697 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:29.792660    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ca4b27-a6bf-445d-8525-9b93ec8ac565" containerID="341ae0202faf1c62f1088a319e3d2277e25961e2facf60cf95c055f9613841a5" exitCode=0
Jul 17 09:27:29.792918 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:29.792718    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerDied","Data":"341ae0202faf1c62f1088a319e3d2277e25961e2facf60cf95c055f9613841a5"}
Jul 17 09:27:30.490207 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:30.490167    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:30.490616 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:30.490169    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:30.490616 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:30.490297    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:30.490616 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:30.490312    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:30.490616 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:30.490399    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:30.490616 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:30.490496    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:31.930742 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:31.930710    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:31.991720 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:31.991681    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:32.489645 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:32.489605    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:32.489881 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:32.489653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:32.489881 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:32.489621    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:32.489881 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:32.489731    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kmjpt" podUID="5bffb080-9b11-4597-ba34-63f4ecfdf75e"
Jul 17 09:27:32.489881 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:32.489794    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7ckzh" podUID="d9c0f5b2-fd40-4bc0-a326-4ca7a466044c"
Jul 17 09:27:32.489881 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:32.489877    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-r6dj8" podUID="00555f83-4bb4-4a88-9bb2-4b427ba80497"
Jul 17 09:27:33.297177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.297146    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-244.ec2.internal" event="NodeReady"
Jul 17 09:27:33.297756 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.297338    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 09:27:33.338114 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.338072    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fm4nh"]
Jul 17 09:27:33.340835 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.340801    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.341009 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.340874    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5pt9b"]
Jul 17 09:27:33.342814 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.342791    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.343109 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.343091    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 09:27:33.343383 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.343362    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 09:27:33.343482 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.343417    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8vfs\""
Jul 17 09:27:33.345305 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.345232    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 09:27:33.345438 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.345310    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 09:27:33.345438 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.345322    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5qcn8\""
Jul 17 09:27:33.345438 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.345270    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 09:27:33.349870 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.349837    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fm4nh"]
Jul 17 09:27:33.352924 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.352856    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pt9b"]
Jul 17 09:27:33.445799 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.445753    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef680662-689d-4cde-8445-9c1322820a32-config-volume\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.445799 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.445793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ccac045-3412-42bb-9aad-ec15a86d870d-cert\") pod \"ingress-canary-5pt9b\" (UID: \"5ccac045-3412-42bb-9aad-ec15a86d870d\") " pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.446060 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.445848    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkwbt\" (UniqueName: \"kubernetes.io/projected/5ccac045-3412-42bb-9aad-ec15a86d870d-kube-api-access-nkwbt\") pod \"ingress-canary-5pt9b\" (UID: \"5ccac045-3412-42bb-9aad-ec15a86d870d\") " pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.446060 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.446002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.446060 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.446036    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tfgf\" (UniqueName: \"kubernetes.io/projected/ef680662-689d-4cde-8445-9c1322820a32-kube-api-access-5tfgf\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.446221 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.446078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef680662-689d-4cde-8445-9c1322820a32-tmp-dir\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.474051 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.474016    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:33.532545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.532512    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:33.546797 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.546766    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.546797 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.546799    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tfgf\" (UniqueName: \"kubernetes.io/projected/ef680662-689d-4cde-8445-9c1322820a32-kube-api-access-5tfgf\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.547043 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.546951    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef680662-689d-4cde-8445-9c1322820a32-tmp-dir\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.547043 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.546994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef680662-689d-4cde-8445-9c1322820a32-config-volume\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.547043 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:33.546959    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:33.547239 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:33.547123    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls podName:ef680662-689d-4cde-8445-9c1322820a32 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:34.047099324 +0000 UTC m=+38.057223244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls") pod "dns-default-fm4nh" (UID: "ef680662-689d-4cde-8445-9c1322820a32") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:33.547239 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.547026    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ccac045-3412-42bb-9aad-ec15a86d870d-cert\") pod \"ingress-canary-5pt9b\" (UID: \"5ccac045-3412-42bb-9aad-ec15a86d870d\") " pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.547239 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.547172    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkwbt\" (UniqueName: \"kubernetes.io/projected/5ccac045-3412-42bb-9aad-ec15a86d870d-kube-api-access-nkwbt\") pod \"ingress-canary-5pt9b\" (UID: \"5ccac045-3412-42bb-9aad-ec15a86d870d\") " pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.547399 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.547353    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef680662-689d-4cde-8445-9c1322820a32-tmp-dir\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.547660 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.547637    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef680662-689d-4cde-8445-9c1322820a32-config-volume\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.552224 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.552186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ccac045-3412-42bb-9aad-ec15a86d870d-cert\") pod \"ingress-canary-5pt9b\" (UID: \"5ccac045-3412-42bb-9aad-ec15a86d870d\") " pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.557773 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.557747    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tfgf\" (UniqueName: \"kubernetes.io/projected/ef680662-689d-4cde-8445-9c1322820a32-kube-api-access-5tfgf\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:33.557773 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.557760    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkwbt\" (UniqueName: \"kubernetes.io/projected/5ccac045-3412-42bb-9aad-ec15a86d870d-kube-api-access-nkwbt\") pod \"ingress-canary-5pt9b\" (UID: \"5ccac045-3412-42bb-9aad-ec15a86d870d\") " pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.662355 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.662308    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pt9b"
Jul 17 09:27:33.964159 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:33.964009    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:34.019330 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.019299    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pt9b"]
Jul 17 09:27:34.022745 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.022673    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:34.022745 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:27:34.022705    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccac045_3412_42bb_9aad_ec15a86d870d.slice/crio-abe4b8504bf01189634b1202d4c7c76395fed5b1e93e80e707586643e7f9d546 WatchSource:0}: Error finding container abe4b8504bf01189634b1202d4c7c76395fed5b1e93e80e707586643e7f9d546: Status 404 returned error can't find the container with id abe4b8504bf01189634b1202d4c7c76395fed5b1e93e80e707586643e7f9d546
Jul 17 09:27:34.051304 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.051271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:34.051464 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:34.051399    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:34.051464 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:34.051453    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls podName:ef680662-689d-4cde-8445-9c1322820a32 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:35.05143873 +0000 UTC m=+39.061562632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls") pod "dns-default-fm4nh" (UID: "ef680662-689d-4cde-8445-9c1322820a32") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:34.489437 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.489341    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:27:34.490196 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.489342    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:27:34.490196 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.489348    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:27:34.492565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.492537    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:27:34.492726 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.492595    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:27:34.492726 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.492657    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 09:27:34.492726 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.492679    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jxtjz\""
Jul 17 09:27:34.492726 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.492680    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 09:27:34.492935 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.492746    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qg8vm\""
Jul 17 09:27:34.809245 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.808606    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ca4b27-a6bf-445d-8525-9b93ec8ac565" containerID="35438ab6930bed0549ec494360a8e58293c7d88718b04c9f58a5fdbf9076a7c3" exitCode=0
Jul 17 09:27:34.809245 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.808723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerDied","Data":"35438ab6930bed0549ec494360a8e58293c7d88718b04c9f58a5fdbf9076a7c3"}
Jul 17 09:27:34.810726 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.810690    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pt9b" event={"ID":"5ccac045-3412-42bb-9aad-ec15a86d870d","Type":"ContainerStarted","Data":"abe4b8504bf01189634b1202d4c7c76395fed5b1e93e80e707586643e7f9d546"}
Jul 17 09:27:34.877035 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.877001    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"]
Jul 17 09:27:34.879422 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.879398    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"
Jul 17 09:27:34.880484 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.880457    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57468f77d6-qnx7c"]
Jul 17 09:27:34.881805 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.881773    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:27:34.881805 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.881798    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d97vj\""
Jul 17 09:27:34.882234 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.882212    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:34.883353 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.883333    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 17 09:27:34.884146 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.884130    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 17 09:27:34.884400 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.884383    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 09:27:34.884964 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.884945    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 17 09:27:34.885088 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.885063    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 17 09:27:34.885145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.885063    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 09:27:34.885253 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.885063    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 17 09:27:34.885363 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.885347    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7bnrl\""
Jul 17 09:27:34.891663 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.891636    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"]
Jul 17 09:27:34.898226 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.898195    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57468f77d6-qnx7c"]
Jul 17 09:27:34.958111 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.957607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgkcs\" (UniqueName: \"kubernetes.io/projected/169e1f61-4f57-45ed-86f7-c6f2a24e96d0-kube-api-access-tgkcs\") pod \"volume-data-source-validator-78cf4c6f6-xk8ql\" (UID: \"169e1f61-4f57-45ed-86f7-c6f2a24e96d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"
Jul 17 09:27:34.958111 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.957673    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:34.958111 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.957699    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-stats-auth\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:34.958111 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.957883    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-default-certificate\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:34.958111 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.957974    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gpk5\" (UniqueName: \"kubernetes.io/projected/f78bfc57-84aa-48a7-910f-24de74875b54-kube-api-access-2gpk5\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:34.958111 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:34.958022    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.025864 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.025799    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:35.058937 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.058894    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.059123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.059001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:35.059123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.059031    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgkcs\" (UniqueName: \"kubernetes.io/projected/169e1f61-4f57-45ed-86f7-c6f2a24e96d0-kube-api-access-tgkcs\") pod \"volume-data-source-validator-78cf4c6f6-xk8ql\" (UID: \"169e1f61-4f57-45ed-86f7-c6f2a24e96d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"
Jul 17 09:27:35.059123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.059068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.059123 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.059081    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 09:27:35.059123 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.059093    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-stats-auth\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.059394 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.059132    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-default-certificate\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.059394 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.059160    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:35.559139266 +0000 UTC m=+39.569263182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : secret "router-metrics-certs-default" not found
Jul 17 09:27:35.059394 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.059169    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:35.059394 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.059205    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gpk5\" (UniqueName: \"kubernetes.io/projected/f78bfc57-84aa-48a7-910f-24de74875b54-kube-api-access-2gpk5\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.059394 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.059214    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:35.559195764 +0000 UTC m=+39.569319668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : configmap references non-existent config key: service-ca.crt
Jul 17 09:27:35.059394 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.059245    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls podName:ef680662-689d-4cde-8445-9c1322820a32 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:37.059234686 +0000 UTC m=+41.069358589 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls") pod "dns-default-fm4nh" (UID: "ef680662-689d-4cde-8445-9c1322820a32") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:35.063068 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.063039    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-stats-auth\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.063211 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.063090    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-default-certificate\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.067272 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.067242    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gpk5\" (UniqueName: \"kubernetes.io/projected/f78bfc57-84aa-48a7-910f-24de74875b54-kube-api-access-2gpk5\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.067418 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.067245    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgkcs\" (UniqueName: \"kubernetes.io/projected/169e1f61-4f57-45ed-86f7-c6f2a24e96d0-kube-api-access-tgkcs\") pod \"volume-data-source-validator-78cf4c6f6-xk8ql\" (UID: \"169e1f61-4f57-45ed-86f7-c6f2a24e96d0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"
Jul 17 09:27:35.124458 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.124421    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:35.193840 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.193775    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"
Jul 17 09:27:35.357085 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.357001    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql"]
Jul 17 09:27:35.361060 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:27:35.361027    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169e1f61_4f57_45ed_86f7_c6f2a24e96d0.slice/crio-9f2fe3b28a1277addf213774b48d9bf4bfc4109ce750fe56d43abbe4208b15ff WatchSource:0}: Error finding container 9f2fe3b28a1277addf213774b48d9bf4bfc4109ce750fe56d43abbe4208b15ff: Status 404 returned error can't find the container with id 9f2fe3b28a1277addf213774b48d9bf4bfc4109ce750fe56d43abbe4208b15ff
Jul 17 09:27:35.563728 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.563687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.564137 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.563750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:35.564137 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.563877    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 09:27:35.564137 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.563889    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:36.563868597 +0000 UTC m=+40.573992500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : configmap references non-existent config key: service-ca.crt
Jul 17 09:27:35.564137 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:35.563917    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:36.563906301 +0000 UTC m=+40.574030204 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : secret "router-metrics-certs-default" not found
Jul 17 09:27:35.814526 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.814489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql" event={"ID":"169e1f61-4f57-45ed-86f7-c6f2a24e96d0","Type":"ContainerStarted","Data":"9f2fe3b28a1277addf213774b48d9bf4bfc4109ce750fe56d43abbe4208b15ff"}
Jul 17 09:27:35.818217 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.818187    2556 generic.go:356] "Generic (PLEG): container finished" podID="35ca4b27-a6bf-445d-8525-9b93ec8ac565" containerID="27d42b7dd1d22e775ba52246a18acbcc25cf30094b80df838da419392adf8b89" exitCode=0
Jul 17 09:27:35.818388 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:35.818232    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerDied","Data":"27d42b7dd1d22e775ba52246a18acbcc25cf30094b80df838da419392adf8b89"}
Jul 17 09:27:36.429109 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:36.429067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:36.479575 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:36.479532    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:36.573956 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:36.573923    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:36.574367 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:36.574009    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:36.574367 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:36.574099    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 09:27:36.574367 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:36.574144    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:38.574126154 +0000 UTC m=+42.584250058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : configmap references non-existent config key: service-ca.crt
Jul 17 09:27:36.574367 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:36.574162    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:38.574154211 +0000 UTC m=+42.584278112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : secret "router-metrics-certs-default" not found
Jul 17 09:27:36.966671 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:36.966623    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:37.067261 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.067226    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:37.078639 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.078601    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:37.078799 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:37.078761    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:37.078884 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:37.078850    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls podName:ef680662-689d-4cde-8445-9c1322820a32 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:41.078807466 +0000 UTC m=+45.088931368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls") pod "dns-default-fm4nh" (UID: "ef680662-689d-4cde-8445-9c1322820a32") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:37.824204 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.824171    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pt9b" event={"ID":"5ccac045-3412-42bb-9aad-ec15a86d870d","Type":"ContainerStarted","Data":"420e2c4b013e16566197d5fe6668ce92b3e44f0182c70c444e8e34eabc9229c1"}
Jul 17 09:27:37.825851 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.825806    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql" event={"ID":"169e1f61-4f57-45ed-86f7-c6f2a24e96d0","Type":"ContainerStarted","Data":"8ee677401688fe6daa4a150f6c5f27ba20d1ca76d861fb4492e6204a0cb7ab45"}
Jul 17 09:27:37.829311 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.829277    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf66q" event={"ID":"35ca4b27-a6bf-445d-8525-9b93ec8ac565","Type":"ContainerStarted","Data":"e142fe2425f53d25290898601397fec67e498435115f6c15f410fbd2ee914353"}
Jul 17 09:27:37.839158 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.839104    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5pt9b" podStartSLOduration=1.858389581 podStartE2EDuration="4.839087574s" totalImagesPullingTime="2.980697993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:37.838659539 +0000 UTC m=+41.848783465" watchObservedRunningTime="2026-07-17 09:27:37.839087574 +0000 UTC m=+41.849211496"
Jul 17 09:27:37.857716 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.857651    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kf66q" podStartSLOduration=8.897396816 podStartE2EDuration="40.857608269s" totalImagesPullingTime="31.960211453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:37.856352322 +0000 UTC m=+41.866476228" watchObservedRunningTime="2026-07-17 09:27:37.857608269 +0000 UTC m=+41.867732194"
Jul 17 09:27:37.873154 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.873073    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xk8ql" podStartSLOduration=2.229988391 podStartE2EDuration="3.873054338s" totalImagesPullingTime="1.643065947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:37.87242965 +0000 UTC m=+41.882553575" watchObservedRunningTime="2026-07-17 09:27:37.873054338 +0000 UTC m=+41.883178263"
Jul 17 09:27:37.977420 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:37.977383    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:38.032154 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:38.032102    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:38.589265 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:38.589214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:38.589422 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:38.589303    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:38.589422 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:38.589381    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 09:27:38.589422 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:38.589408    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.589393854 +0000 UTC m=+46.599517756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : configmap references non-existent config key: service-ca.crt
Jul 17 09:27:38.589546 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:38.589436    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.589423767 +0000 UTC m=+46.599547669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : secret "router-metrics-certs-default" not found
Jul 17 09:27:38.926866 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:38.926809    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:39.000430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.000396    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:39.056878 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.056841    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:39.440591 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.440557    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"]
Jul 17 09:27:39.471163 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.471125    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"]
Jul 17 09:27:39.471322 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.471220    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"
Jul 17 09:27:39.473423 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.473397    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cjxd4\""
Jul 17 09:27:39.597619 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.597574    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9vbk\" (UniqueName: \"kubernetes.io/projected/1759c263-26ce-4482-8cb4-33e243c8e3bc-kube-api-access-h9vbk\") pod \"network-check-source-7bc5b6bf85-t977w\" (UID: \"1759c263-26ce-4482-8cb4-33e243c8e3bc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"
Jul 17 09:27:39.698565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.698495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9vbk\" (UniqueName: \"kubernetes.io/projected/1759c263-26ce-4482-8cb4-33e243c8e3bc-kube-api-access-h9vbk\") pod \"network-check-source-7bc5b6bf85-t977w\" (UID: \"1759c263-26ce-4482-8cb4-33e243c8e3bc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"
Jul 17 09:27:39.706900 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.706864    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9vbk\" (UniqueName: \"kubernetes.io/projected/1759c263-26ce-4482-8cb4-33e243c8e3bc-kube-api-access-h9vbk\") pod \"network-check-source-7bc5b6bf85-t977w\" (UID: \"1759c263-26ce-4482-8cb4-33e243c8e3bc\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"
Jul 17 09:27:39.780935 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.780891    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"
Jul 17 09:27:39.933446 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:39.933388    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w"]
Jul 17 09:27:39.936838 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:27:39.936791    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1759c263_26ce_4482_8cb4_33e243c8e3bc.slice/crio-899498ebfdc5280cbc7723f4125273f9c51ce9fdcccdd1f2a9cc2cbb0e799099 WatchSource:0}: Error finding container 899498ebfdc5280cbc7723f4125273f9c51ce9fdcccdd1f2a9cc2cbb0e799099: Status 404 returned error can't find the container with id 899498ebfdc5280cbc7723f4125273f9c51ce9fdcccdd1f2a9cc2cbb0e799099
Jul 17 09:27:40.838410 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:40.838373    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w" event={"ID":"1759c263-26ce-4482-8cb4-33e243c8e3bc","Type":"ContainerStarted","Data":"899498ebfdc5280cbc7723f4125273f9c51ce9fdcccdd1f2a9cc2cbb0e799099"}
Jul 17 09:27:41.066102 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.066067    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jgxft"]
Jul 17 09:27:41.079714 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.079680    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.083207 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.083174    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 09:27:41.083698 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.083678    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gmxxn\""
Jul 17 09:27:41.084317 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.084072    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 09:27:41.084317 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.084092    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 09:27:41.084317 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.084105    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 09:27:41.099171 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.099089    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jgxft"]
Jul 17 09:27:41.109405 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.109368    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:41.109587 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.109508    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:41.109587 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.109574    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls podName:ef680662-689d-4cde-8445-9c1322820a32 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:49.10955487 +0000 UTC m=+53.119678772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls") pod "dns-default-fm4nh" (UID: "ef680662-689d-4cde-8445-9c1322820a32") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:41.127392 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.127362    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:41.191109 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.191072    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:41.210160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.210108    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.210330 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.210179    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k57hl\" (UniqueName: \"kubernetes.io/projected/914869ab-9df5-45b6-962a-e98f8804de31-kube-api-access-k57hl\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.210330 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.210214    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/914869ab-9df5-45b6-962a-e98f8804de31-crio-socket\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.210330 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.210245    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/914869ab-9df5-45b6-962a-e98f8804de31-data-volume\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.210330 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.210274    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.310642 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.310592    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.310868 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.310654    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k57hl\" (UniqueName: \"kubernetes.io/projected/914869ab-9df5-45b6-962a-e98f8804de31-kube-api-access-k57hl\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.310868 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.310688    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/914869ab-9df5-45b6-962a-e98f8804de31-crio-socket\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.310868 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.310716    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/914869ab-9df5-45b6-962a-e98f8804de31-data-volume\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.310868 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.310746    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.310868 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.310754    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:41.310868 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.310845    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:41.810805505 +0000 UTC m=+45.820929409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:41.311115 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.310884    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.311115 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.310944    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:41.810928876 +0000 UTC m=+45.821052779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.311115 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.310951    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/914869ab-9df5-45b6-962a-e98f8804de31-crio-socket\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.311115 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.311104    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/914869ab-9df5-45b6-962a-e98f8804de31-data-volume\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.320871 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.320832    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k57hl\" (UniqueName: \"kubernetes.io/projected/914869ab-9df5-45b6-962a-e98f8804de31-kube-api-access-k57hl\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.397261 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.397214    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:41.496316 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.496285    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:41.815194 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.815100    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.815366 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:41.815201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:41.815366 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.815228    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.815366 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.815305    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:41.815366 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.815316    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.815295193 +0000 UTC m=+46.825419108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.815366 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:41.815345    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.815333901 +0000 UTC m=+46.825457803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:42.139247 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.139216    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:42.239893 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.239858    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:42.425937 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.425806    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:42.621614 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.621581    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:42.621968 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.621674    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:42.621968 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.621774    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:50.62175574 +0000 UTC m=+54.631879647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : configmap references non-existent config key: service-ca.crt
Jul 17 09:27:42.621968 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.621867    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 09:27:42.621968 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.621930    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:50.621914102 +0000 UTC m=+54.632038003 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : secret "router-metrics-certs-default" not found
Jul 17 09:27:42.778318 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.778222    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:42.823422 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.823387    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:42.823611 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.823511    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:42.823611 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:42.823510    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:42.823611 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.823574    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:44.823555561 +0000 UTC m=+48.833679463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:42.823611 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.823583    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:42.823764 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:42.823646    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:44.82360883 +0000 UTC m=+48.833732735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:43.700138 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.700098    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rv2qx"]
Jul 17 09:27:43.731269 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.731210    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rv2qx"]
Jul 17 09:27:43.731469 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.731361    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.733771 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.733739    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 17 09:27:43.733941 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.733739    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 17 09:27:43.734775 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.734509    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 17 09:27:43.734775 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.734531    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 17 09:27:43.734775 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.734513    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bb9zk\""
Jul 17 09:27:43.737852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.737725    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:43.792614 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.792578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:43.832157 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.832122    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-signing-key\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.832366 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.832336    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-signing-cabundle\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.832428 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.832383    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jjnx\" (UniqueName: \"kubernetes.io/projected/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-kube-api-access-7jjnx\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.836768 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.836743    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rb5ff_01e70b2c-a732-4ba8-b436-3caa5b43ccca/dns-node-resolver/0.log"
Jul 17 09:27:43.933635 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.933593    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-signing-cabundle\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.933635 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.933627    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jjnx\" (UniqueName: \"kubernetes.io/projected/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-kube-api-access-7jjnx\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.933896 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.933760    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-signing-key\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.934391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.934369    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-signing-cabundle\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.936289 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.936271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-signing-key\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:43.941214 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:43.941189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jjnx\" (UniqueName: \"kubernetes.io/projected/ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1-kube-api-access-7jjnx\") pod \"service-ca-58598695f7-rv2qx\" (UID: \"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1\") " pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:44.043992 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.043901    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-rv2qx"
Jul 17 09:27:44.186744 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.186693    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rv2qx"]
Jul 17 09:27:44.320485 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:27:44.320408    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2c7313_3785_4fbe_b6d1_2fe3a9fde7f1.slice/crio-41995068fd9e5071ee93a55ff3abc7f98f08f38e1188d433151e605c6839f8dc WatchSource:0}: Error finding container 41995068fd9e5071ee93a55ff3abc7f98f08f38e1188d433151e605c6839f8dc: Status 404 returned error can't find the container with id 41995068fd9e5071ee93a55ff3abc7f98f08f38e1188d433151e605c6839f8dc
Jul 17 09:27:44.635959 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.635928    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dqwl4_82773457-e021-4aa2-84e9-7d1fe8045d0c/node-ca/0.log"
Jul 17 09:27:44.843368 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.843325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:44.843804 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.843401    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:44.843804 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:44.843494    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:44.843804 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:44.843511    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:44.843804 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:44.843556    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.843538709 +0000 UTC m=+52.853662611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:44.843804 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:44.843571    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.843563409 +0000 UTC m=+52.853687311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:44.850753 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.850704    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w" event={"ID":"1759c263-26ce-4482-8cb4-33e243c8e3bc","Type":"ContainerStarted","Data":"765f2ef7213a0729404a110cb441f2a66e9a630e862b0fe50a716885e8e527bc"}
Jul 17 09:27:44.852336 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.852299    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-rv2qx" event={"ID":"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1","Type":"ContainerStarted","Data":"41995068fd9e5071ee93a55ff3abc7f98f08f38e1188d433151e605c6839f8dc"}
Jul 17 09:27:44.869786 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.869718    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t977w" podStartSLOduration=1.457856407 podStartE2EDuration="5.869698162s" totalImagesPullingTime="4.411841755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:44.869465034 +0000 UTC m=+48.879588959" watchObservedRunningTime="2026-07-17 09:27:44.869698162 +0000 UTC m=+48.879822089"
Jul 17 09:27:44.913513 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:44.913424    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:45.236846 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:45.236735    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pt9b_5ccac045-3412-42bb-9aad-ec15a86d870d/serve-healthcheck-canary/0.log"
Jul 17 09:27:46.859343 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:46.859255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-rv2qx" event={"ID":"ae2c7313-3785-4fbe-b6d1-2fe3a9fde7f1","Type":"ContainerStarted","Data":"4583cbe87d03862085f9586a358289f00020da73f68c0d57d511cb78a49c0826"}
Jul 17 09:27:46.874722 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:46.874666    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-rv2qx" podStartSLOduration=1.640629946 podStartE2EDuration="3.874649519s" totalImagesPullingTime="2.234019573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:46.873343692 +0000 UTC m=+50.883467617" watchObservedRunningTime="2026-07-17 09:27:46.874649519 +0000 UTC m=+50.884773443"
Jul 17 09:27:47.306558 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:47.306507    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.020293 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.020258    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.122140 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.122109    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.203005 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.202975    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.283021 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.282923    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.382588 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.382552    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.539694 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.539541    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.724300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.724241    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:48.882655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.882612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:48.882900 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:48.882744    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:48.882900 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:48.882748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:48.882900 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:48.882816    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:56.882795457 +0000 UTC m=+60.892919367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:48.882900 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:48.882865    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:48.883088 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:48.882917    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:56.882900321 +0000 UTC m=+60.893024227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:49.184616 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:49.184528    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:27:49.185080 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:49.184692    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:49.185080 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:49.184762    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls podName:ef680662-689d-4cde-8445-9c1322820a32 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:05.184747079 +0000 UTC m=+69.194870981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls") pod "dns-default-fm4nh" (UID: "ef680662-689d-4cde-8445-9c1322820a32") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:49.716174 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:49.716135    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:50.116445 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.116353    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:50.317938 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.317899    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:50.518225 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.518194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:50.698124 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.698081    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:50.698306 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.698155    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:27:50.698306 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:50.698283    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 09:27:50.698306 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:50.698289    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:06.698265659 +0000 UTC m=+70.708389565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : configmap references non-existent config key: service-ca.crt
Jul 17 09:27:50.698424 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:50.698329    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs podName:f78bfc57-84aa-48a7-910f-24de74875b54 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:06.698316125 +0000 UTC m=+70.708440027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs") pod "router-default-57468f77d6-qnx7c" (UID: "f78bfc57-84aa-48a7-910f-24de74875b54") : secret "router-metrics-certs-default" not found
Jul 17 09:27:50.722251 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.722215    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:50.778682 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:50.778610    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:51.051525 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:51.051438    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:51.708817 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:51.708780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:51.760877 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:51.760847    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:52.066361 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:52.066278    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:52.161497 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:52.161464    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:52.314108 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:52.314067    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:52.515295 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:52.515262    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:52.716630 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:52.716597    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:52.767395 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:52.767313    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:53.720188 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:53.720146    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:54.114716 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:54.114617    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:54.316595 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:54.316556    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:54.512855 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:54.512799    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:54.914077 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:54.914043    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:55.306161 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:55.306079    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:55.513450 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:55.513410    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:55.921410 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:55.921374    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:56.122946 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:56.122911    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:56.534790 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:56.534760    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:56.946051 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:56.946012    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:56.946441 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:56.946106    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:56.946441 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:56.946192    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:56.946441 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:27:56.946253    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls podName:914869ab-9df5-45b6-962a-e98f8804de31 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:12.946239173 +0000 UTC m=+76.956363075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jgxft" (UID: "914869ab-9df5-45b6-962a-e98f8804de31") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:56.947172 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:56.947148    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/914869ab-9df5-45b6-962a-e98f8804de31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:27:57.378204 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:57.378171    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:57.737519 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:57.737439    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:57.944702 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:57.944657    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:58.121180 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:58.121151    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:58.320457 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:58.320421    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:58.722556 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:58.722526    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:58.808528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:58.808496    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9qd4q"
Jul 17 09:27:59.976705 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:27:59.976670    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:00.088610 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:00.088576    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:00.186816 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:00.186783    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:00.264246 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:00.264159    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:00.436995 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:00.436963    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:00.632895 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:00.632868    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:01.280126 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.280075    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:28:01.280586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.280197    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:28:01.283067 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.283046    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:28:01.283601 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.283584    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:28:01.293187 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.293164    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/00555f83-4bb4-4a88-9bb2-4b427ba80497-original-pull-secret\") pod \"global-pull-secret-syncer-r6dj8\" (UID: \"00555f83-4bb4-4a88-9bb2-4b427ba80497\") " pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:28:01.293284 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.293163    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9c0f5b2-fd40-4bc0-a326-4ca7a466044c-metrics-certs\") pod \"network-metrics-daemon-7ckzh\" (UID: \"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c\") " pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:28:01.380923 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.380869    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:28:01.383458 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.383436    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4svj\" (UniqueName: \"kubernetes.io/projected/5bffb080-9b11-4597-ba34-63f4ecfdf75e-kube-api-access-g4svj\") pod \"network-check-target-kmjpt\" (UID: \"5bffb080-9b11-4597-ba34-63f4ecfdf75e\") " pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:28:01.503655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.503623    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jxtjz\""
Jul 17 09:28:01.511804 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.511771    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7ckzh"
Jul 17 09:28:01.512403 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.512384    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qg8vm\""
Jul 17 09:28:01.516098 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.516077    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-r6dj8"
Jul 17 09:28:01.519903 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.519883    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:28:01.678254 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.678179    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:01.678993 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.678930    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kmjpt"]
Jul 17 09:28:01.683225 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:01.683186    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bffb080_9b11_4597_ba34_63f4ecfdf75e.slice/crio-1af2e3847ac9ccdd77a5a370234c897a2e7810244fbf6a1965660efc0491891f WatchSource:0}: Error finding container 1af2e3847ac9ccdd77a5a370234c897a2e7810244fbf6a1965660efc0491891f: Status 404 returned error can't find the container with id 1af2e3847ac9ccdd77a5a370234c897a2e7810244fbf6a1965660efc0491891f
Jul 17 09:28:01.684343 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.684266    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7ckzh"]
Jul 17 09:28:01.686859 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:01.686709    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c0f5b2_fd40_4bc0_a326_4ca7a466044c.slice/crio-11854f54aa6b3ee8fd26c8194b4c81e8ad78870cca6507f975808c8de9ce76b4 WatchSource:0}: Error finding container 11854f54aa6b3ee8fd26c8194b4c81e8ad78870cca6507f975808c8de9ce76b4: Status 404 returned error can't find the container with id 11854f54aa6b3ee8fd26c8194b4c81e8ad78870cca6507f975808c8de9ce76b4
Jul 17 09:28:01.698389 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.698357    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-r6dj8"]
Jul 17 09:28:01.701205 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:01.701167    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00555f83_4bb4_4a88_9bb2_4b427ba80497.slice/crio-25590e2c76a14125a9e2b34671efab9fbf33b2e171ff60584c7809e73e46d24e WatchSource:0}: Error finding container 25590e2c76a14125a9e2b34671efab9fbf33b2e171ff60584c7809e73e46d24e: Status 404 returned error can't find the container with id 25590e2c76a14125a9e2b34671efab9fbf33b2e171ff60584c7809e73e46d24e
Jul 17 09:28:01.775324 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.775288    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:01.877741 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.877710    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:01.904298 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.904265    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r6dj8" event={"ID":"00555f83-4bb4-4a88-9bb2-4b427ba80497","Type":"ContainerStarted","Data":"25590e2c76a14125a9e2b34671efab9fbf33b2e171ff60584c7809e73e46d24e"}
Jul 17 09:28:01.905548 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.905521    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ckzh" event={"ID":"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c","Type":"ContainerStarted","Data":"11854f54aa6b3ee8fd26c8194b4c81e8ad78870cca6507f975808c8de9ce76b4"}
Jul 17 09:28:01.907018 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.906992    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kmjpt" event={"ID":"5bffb080-9b11-4597-ba34-63f4ecfdf75e","Type":"ContainerStarted","Data":"9f7536c198869e709048813bf75658105fe990817fe5aecb5a8b7cef6df9de35"}
Jul 17 09:28:01.907131 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.907032    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kmjpt" event={"ID":"5bffb080-9b11-4597-ba34-63f4ecfdf75e","Type":"ContainerStarted","Data":"1af2e3847ac9ccdd77a5a370234c897a2e7810244fbf6a1965660efc0491891f"}
Jul 17 09:28:01.907131 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.907124    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:28:01.930526 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.930475    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kmjpt" podStartSLOduration=64.930457469 podStartE2EDuration="1m4.930457469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:01.929750264 +0000 UTC m=+65.939874188" watchObservedRunningTime="2026-07-17 09:28:01.930457469 +0000 UTC m=+65.940581392"
Jul 17 09:28:01.983620 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:01.983584    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:02.061558 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:02.061526    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:03.916337 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:03.916296    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ckzh" event={"ID":"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c","Type":"ContainerStarted","Data":"63afa45961697aa5fd36bef3b854c559c1bb5a985c5778830b6e0d1a8f416397"}
Jul 17 09:28:03.916779 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:03.916372    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7ckzh" event={"ID":"d9c0f5b2-fd40-4bc0-a326-4ca7a466044c","Type":"ContainerStarted","Data":"d54235e014f295953b3c520d46b1750fd1d324090f497bdd27bf6c2b9cd31742"}
Jul 17 09:28:03.934540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:03.934470    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7ckzh" podStartSLOduration=65.566995726 podStartE2EDuration="1m6.934444847s" totalImagesPullingTime="1.367449121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:03.932439311 +0000 UTC m=+67.942563230" watchObservedRunningTime="2026-07-17 09:28:03.934444847 +0000 UTC m=+67.944568773"
Jul 17 09:28:04.597462 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.597422    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"]
Jul 17 09:28:04.601239 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.601209    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"]
Jul 17 09:28:04.601381 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.601351    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.604815 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.604783    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.605907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.605882    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 09:28:04.606053 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.605948    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6k6nc\""
Jul 17 09:28:04.606120 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.606091    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 09:28:04.608689 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.608663    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 09:28:04.608878 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.608757    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 17 09:28:04.617079 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.617054    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 17 09:28:04.617496 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.617459    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 09:28:04.617496 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.617487    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 09:28:04.617648 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.617535    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 17 09:28:04.617648 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.617596    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 17 09:28:04.623944 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.623897    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"]
Jul 17 09:28:04.625105 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.625078    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"]
Jul 17 09:28:04.708811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.708774    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gjkh\" (UniqueName: \"kubernetes.io/projected/b3cb61ad-debb-4714-85f1-fe8db2966b31-kube-api-access-5gjkh\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.708811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.708845    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.709087 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.708886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/960c6dec-6cfb-4169-bf6e-2ec010ecae40-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-fqb94\" (UID: \"960c6dec-6cfb-4169-bf6e-2ec010ecae40\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.709087 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.708902    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.709087 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.708963    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-ca\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.709087 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.708997    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-hub\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.709087 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.709034    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb61ad-debb-4714-85f1-fe8db2966b31-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.709087 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.709067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960c6dec-6cfb-4169-bf6e-2ec010ecae40-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-fqb94\" (UID: \"960c6dec-6cfb-4169-bf6e-2ec010ecae40\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.810094 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810053    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810105    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/960c6dec-6cfb-4169-bf6e-2ec010ecae40-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-fqb94\" (UID: \"960c6dec-6cfb-4169-bf6e-2ec010ecae40\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810145    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-ca\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810160    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-hub\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb61ad-debb-4714-85f1-fe8db2966b31-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960c6dec-6cfb-4169-bf6e-2ec010ecae40-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-fqb94\" (UID: \"960c6dec-6cfb-4169-bf6e-2ec010ecae40\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.810300 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.810272    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gjkh\" (UniqueName: \"kubernetes.io/projected/b3cb61ad-debb-4714-85f1-fe8db2966b31-kube-api-access-5gjkh\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.811051 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.811024    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3cb61ad-debb-4714-85f1-fe8db2966b31-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.811176 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.811115    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/960c6dec-6cfb-4169-bf6e-2ec010ecae40-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-fqb94\" (UID: \"960c6dec-6cfb-4169-bf6e-2ec010ecae40\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.812903 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.812877    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-ca\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.813545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.813518    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.813638 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.813525    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.813638 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.813547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b3cb61ad-debb-4714-85f1-fe8db2966b31-hub\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.813638 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.813604    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/960c6dec-6cfb-4169-bf6e-2ec010ecae40-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-fqb94\" (UID: \"960c6dec-6cfb-4169-bf6e-2ec010ecae40\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.821647 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.821617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gjkh\" (UniqueName: \"kubernetes.io/projected/b3cb61ad-debb-4714-85f1-fe8db2966b31-kube-api-access-5gjkh\") pod \"cluster-proxy-proxy-agent-54fbd48c88-spcmn\" (UID: \"b3cb61ad-debb-4714-85f1-fe8db2966b31\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.853064 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.852974    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:04.913378 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.913344    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"
Jul 17 09:28:04.930983 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.930937    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"
Jul 17 09:28:04.958291 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:04.958255    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:05.091919 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:05.091882    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:05.213161 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:05.213122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:28:05.216117 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:05.216092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef680662-689d-4cde-8445-9c1322820a32-metrics-tls\") pod \"dns-default-fm4nh\" (UID: \"ef680662-689d-4cde-8445-9c1322820a32\") " pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:28:05.458616 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:05.458582    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8vfs\""
Jul 17 09:28:05.466270 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:05.466176    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:28:06.430377 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.430340    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fm4nh"]
Jul 17 09:28:06.585454 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.585362    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:06.594411 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:06.594382    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef680662_689d_4cde_8445_9c1322820a32.slice/crio-80468035dda6ab812454236b43fc0119e7dfe5c712336cca17addfc241049209 WatchSource:0}: Error finding container 80468035dda6ab812454236b43fc0119e7dfe5c712336cca17addfc241049209: Status 404 returned error can't find the container with id 80468035dda6ab812454236b43fc0119e7dfe5c712336cca17addfc241049209
Jul 17 09:28:06.651574 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.651538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94"]
Jul 17 09:28:06.656070 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:06.655194    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960c6dec_6cfb_4169_bf6e_2ec010ecae40.slice/crio-9845fb004bce4f195eedcbcd7e269576d14d1a017862c1cda43acd7b2ba60fb5 WatchSource:0}: Error finding container 9845fb004bce4f195eedcbcd7e269576d14d1a017862c1cda43acd7b2ba60fb5: Status 404 returned error can't find the container with id 9845fb004bce4f195eedcbcd7e269576d14d1a017862c1cda43acd7b2ba60fb5
Jul 17 09:28:06.656070 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.655548    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:06.656070 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.655711    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn"]
Jul 17 09:28:06.659109 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:06.659083    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3cb61ad_debb_4714_85f1_fe8db2966b31.slice/crio-94e7b63165b20ab11d15ca5530011baa2fb01288a3843f08144038296057602f WatchSource:0}: Error finding container 94e7b63165b20ab11d15ca5530011baa2fb01288a3843f08144038296057602f: Status 404 returned error can't find the container with id 94e7b63165b20ab11d15ca5530011baa2fb01288a3843f08144038296057602f
Jul 17 09:28:06.724506 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.724472    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:06.724672 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.724541    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:06.725149 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.725126    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f78bfc57-84aa-48a7-910f-24de74875b54-service-ca-bundle\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:06.726721 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.726704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f78bfc57-84aa-48a7-910f-24de74875b54-metrics-certs\") pod \"router-default-57468f77d6-qnx7c\" (UID: \"f78bfc57-84aa-48a7-910f-24de74875b54\") " pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:06.929080 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.929049    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-r6dj8" event={"ID":"00555f83-4bb4-4a88-9bb2-4b427ba80497","Type":"ContainerStarted","Data":"f24b647802d1e324262c2a3e8beb5b5799e4481dac723334514ef9f5543f20b9"}
Jul 17 09:28:06.930367 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.930332    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fm4nh" event={"ID":"ef680662-689d-4cde-8445-9c1322820a32","Type":"ContainerStarted","Data":"80468035dda6ab812454236b43fc0119e7dfe5c712336cca17addfc241049209"}
Jul 17 09:28:06.931506 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.931480    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn" event={"ID":"b3cb61ad-debb-4714-85f1-fe8db2966b31","Type":"ContainerStarted","Data":"94e7b63165b20ab11d15ca5530011baa2fb01288a3843f08144038296057602f"}
Jul 17 09:28:06.932701 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.932673    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94" event={"ID":"960c6dec-6cfb-4169-bf6e-2ec010ecae40","Type":"ContainerStarted","Data":"9845fb004bce4f195eedcbcd7e269576d14d1a017862c1cda43acd7b2ba60fb5"}
Jul 17 09:28:06.944564 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:06.944502    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-r6dj8" podStartSLOduration=65.017657989 podStartE2EDuration="1m9.944479938s" totalImagesPullingTime="4.926821949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:06.943956853 +0000 UTC m=+70.954080778" watchObservedRunningTime="2026-07-17 09:28:06.944479938 +0000 UTC m=+70.954603866"
Jul 17 09:28:07.012488 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:07.012452    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7bnrl\""
Jul 17 09:28:07.020588 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:07.020554    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:07.235605 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:07.235520    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57468f77d6-qnx7c"]
Jul 17 09:28:07.239128 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:07.239027    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78bfc57_84aa_48a7_910f_24de74875b54.slice/crio-1ea519f3bec4044fcb5b174148b4d51f46c5f4b79da09348f888bc3ab7764ac7 WatchSource:0}: Error finding container 1ea519f3bec4044fcb5b174148b4d51f46c5f4b79da09348f888bc3ab7764ac7: Status 404 returned error can't find the container with id 1ea519f3bec4044fcb5b174148b4d51f46c5f4b79da09348f888bc3ab7764ac7
Jul 17 09:28:07.942790 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:07.942755    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57468f77d6-qnx7c" event={"ID":"f78bfc57-84aa-48a7-910f-24de74875b54","Type":"ContainerStarted","Data":"c89fe4dc530c97bac22915dcd44ad863755bcfdad92da43c75ee6b1829a6a0fc"}
Jul 17 09:28:07.943279 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:07.942810    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57468f77d6-qnx7c" event={"ID":"f78bfc57-84aa-48a7-910f-24de74875b54","Type":"ContainerStarted","Data":"1ea519f3bec4044fcb5b174148b4d51f46c5f4b79da09348f888bc3ab7764ac7"}
Jul 17 09:28:08.021708 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.021628    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:08.025682 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.025460    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:08.045217 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.045155    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57468f77d6-qnx7c" podStartSLOduration=34.045132683 podStartE2EDuration="34.045132683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:07.961179125 +0000 UTC m=+71.971303049" watchObservedRunningTime="2026-07-17 09:28:08.045132683 +0000 UTC m=+72.055256607"
Jul 17 09:28:08.298540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.298447    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:08.366759 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.366725    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:08.946167 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.945922    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:08.948105 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:08.948078    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57468f77d6-qnx7c"
Jul 17 09:28:10.872951 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.872680    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:10.956391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.955402    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn" event={"ID":"b3cb61ad-debb-4714-85f1-fe8db2966b31","Type":"ContainerStarted","Data":"f696cf94b01cbe645050aa33081b2ef11a53bfbfc6a0d53fc445ac0b28c7095d"}
Jul 17 09:28:10.956391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.955459    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn" event={"ID":"b3cb61ad-debb-4714-85f1-fe8db2966b31","Type":"ContainerStarted","Data":"9d1727c93571b9a3dc017476b6a08860fea7764df79fdbb822c61dd3da0c70b3"}
Jul 17 09:28:10.956987 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.956954    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94" event={"ID":"960c6dec-6cfb-4169-bf6e-2ec010ecae40","Type":"ContainerStarted","Data":"cca277548e57e197ce72d9e65872175982b41f68b288c78578d3144c6ae102e6"}
Jul 17 09:28:10.959774 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.959736    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fm4nh" event={"ID":"ef680662-689d-4cde-8445-9c1322820a32","Type":"ContainerStarted","Data":"61bad376e63e45f4087c8920c8de99670e22ad78e0e70d873e3c30db7fd9a457"}
Jul 17 09:28:10.959959 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.959791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fm4nh" event={"ID":"ef680662-689d-4cde-8445-9c1322820a32","Type":"ContainerStarted","Data":"c3d15363aa0c5e3de6d53bdbad0d40b2d663d76d95085a5eeed6e46794809469"}
Jul 17 09:28:10.973238 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.973083    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-fqb94" podStartSLOduration=2.912827637 podStartE2EDuration="6.973062214s" totalImagesPullingTime="4.060234577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:10.973010233 +0000 UTC m=+74.983134157" watchObservedRunningTime="2026-07-17 09:28:10.973062214 +0000 UTC m=+74.983186130"
Jul 17 09:28:10.976705 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.976672    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:10.991689 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:10.991564    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fm4nh" podStartSLOduration=33.883584812 podStartE2EDuration="37.991524315s" totalImagesPullingTime="4.107939503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:10.990815386 +0000 UTC m=+75.000939379" watchObservedRunningTime="2026-07-17 09:28:10.991524315 +0000 UTC m=+75.001648242"
Jul 17 09:28:11.086974 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:11.086937    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:11.146361 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:11.146324    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:11.965372 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:11.965327    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn" event={"ID":"b3cb61ad-debb-4714-85f1-fe8db2966b31","Type":"ContainerStarted","Data":"b5d14440dc7b48c209a788c8b02aba81fd0c8ecbbfac9de74769166ef9479376"}
Jul 17 09:28:11.965866 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:11.965809    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:28:11.985991 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:11.985933    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54fbd48c88-spcmn" podStartSLOduration=3.8666841400000003 podStartE2EDuration="7.985913372s" totalImagesPullingTime="4.119229232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:11.985161245 +0000 UTC m=+75.995285170" watchObservedRunningTime="2026-07-17 09:28:11.985913372 +0000 UTC m=+75.996037296"
Jul 17 09:28:12.989972 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:12.989925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:28:12.992550 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:12.992521    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/914869ab-9df5-45b6-962a-e98f8804de31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jgxft\" (UID: \"914869ab-9df5-45b6-962a-e98f8804de31\") " pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:28:13.193684 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:13.193646    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gmxxn\""
Jul 17 09:28:13.201520 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:13.201477    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jgxft"
Jul 17 09:28:13.344184 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:13.344150    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jgxft"]
Jul 17 09:28:13.347673 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:13.347643    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914869ab_9df5_45b6_962a_e98f8804de31.slice/crio-d2e089bdb60e70d1164758a62b53c18a2b6a62c17ba1bb1922050731e5e9241f WatchSource:0}: Error finding container d2e089bdb60e70d1164758a62b53c18a2b6a62c17ba1bb1922050731e5e9241f: Status 404 returned error can't find the container with id d2e089bdb60e70d1164758a62b53c18a2b6a62c17ba1bb1922050731e5e9241f
Jul 17 09:28:13.977183 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:13.977137    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgxft" event={"ID":"914869ab-9df5-45b6-962a-e98f8804de31","Type":"ContainerStarted","Data":"d2e089bdb60e70d1164758a62b53c18a2b6a62c17ba1bb1922050731e5e9241f"}
Jul 17 09:28:14.604585 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:14.604547    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:14.781347 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:14.781304    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:15.250641 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:15.250596    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:19.000096 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:19.000049    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgxft" event={"ID":"914869ab-9df5-45b6-962a-e98f8804de31","Type":"ContainerStarted","Data":"bfccf522e8db0922794b2f622e72c23795dc02ed38e376cd077f31302043d4bd"}
Jul 17 09:28:20.074588 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:20.074547    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:20.154044 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:20.154009    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:21.971025 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:21.970997    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fm4nh"
Jul 17 09:28:22.012497 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:22.012466    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgxft" event={"ID":"914869ab-9df5-45b6-962a-e98f8804de31","Type":"ContainerStarted","Data":"2797051f374f1d071a1c74e2def81165b8d30257d222710c08b80abd2cfbc12d"}
Jul 17 09:28:23.464110 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:23.464074    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:23.565770 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:23.565734    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:26.028484 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:26.028422    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jgxft" event={"ID":"914869ab-9df5-45b6-962a-e98f8804de31","Type":"ContainerStarted","Data":"beb004cb739d4d79c6a66eafb96d8dcf1207b02e51e19a5f7e3e5ea7d06db16a"}
Jul 17 09:28:26.045243 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:26.045168    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jgxft" podStartSLOduration=33.178353696 podStartE2EDuration="45.045150497s" totalImagesPullingTime="11.866796801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:27:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:26.044016468 +0000 UTC m=+90.054140390" watchObservedRunningTime="2026-07-17 09:28:26.045150497 +0000 UTC m=+90.055274421"
Jul 17 09:28:28.574769 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:28.574737    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:28.675654 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:28.675627    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:28.726718 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:28.726684    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:30.268703 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:30.268662    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:30.327779 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:30.327742    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:31.101245 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:31.101211    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:31.201329 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:31.201298    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:31.871413 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:31.871381    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:31.972719 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:31.972689    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:32.457809 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:32.457768    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:32.522535 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:32.522498    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:32.809078 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:32.808990    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:32.881016 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:32.880984    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:32.913102 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:32.913069    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kmjpt"
Jul 17 09:28:33.078336 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:33.078253    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:33.180138 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:33.180097    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:37.499405 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:37.499358    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:38.112699 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:38.112665    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:38.171043 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:38.171018    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:38.252554 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:38.252520    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:38.352446 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:38.352407    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:39.934314 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:39.934271    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.147268 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.147219    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.191450 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.191351    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l2gjk"]
Jul 17 09:28:40.194969 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.194943    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.198056 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.198028    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 09:28:40.198215 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.198029    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 09:28:40.198854 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.198805    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 09:28:40.198953 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.198870    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 09:28:40.198953 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.198888    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 09:28:40.199059 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.198808    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pf7jj\""
Jul 17 09:28:40.199059 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.199052    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 09:28:40.265747 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.265716    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.306353 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306307    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-sys\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306353 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306351    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-textfile\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306444    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-wtmp\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306491    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebb15d5e-7a58-43af-ab73-9c7843099256-metrics-client-ca\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306530    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306586 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306560    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-accelerators-collector-config\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306746 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306588    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-tls\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306746 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306643    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-root\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.306746 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.306680    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bhrr\" (UniqueName: \"kubernetes.io/projected/ebb15d5e-7a58-43af-ab73-9c7843099256-kube-api-access-2bhrr\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.326565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.326528    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.407889 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.407845    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-root\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.407908    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bhrr\" (UniqueName: \"kubernetes.io/projected/ebb15d5e-7a58-43af-ab73-9c7843099256-kube-api-access-2bhrr\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.407945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-sys\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.407971    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-textfile\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.407976    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-root\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408016    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-wtmp\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408040    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebb15d5e-7a58-43af-ab73-9c7843099256-metrics-client-ca\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408045    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-sys\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408067    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408094    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-accelerators-collector-config\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408620 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-tls\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.408620 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:28:40.408249    2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 09:28:40.408620 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:28:40.408312    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-tls podName:ebb15d5e-7a58-43af-ab73-9c7843099256 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:40.908289864 +0000 UTC m=+104.918413771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-tls") pod "node-exporter-l2gjk" (UID: "ebb15d5e-7a58-43af-ab73-9c7843099256") : secret "node-exporter-tls" not found
Jul 17 09:28:40.408775 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.408676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-textfile\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.409039 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.409015    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-wtmp\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.409219 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.409194    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-accelerators-collector-config\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.409594 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.409573    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ebb15d5e-7a58-43af-ab73-9c7843099256-metrics-client-ca\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.411806 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.411780    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.417546 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.417504    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bhrr\" (UniqueName: \"kubernetes.io/projected/ebb15d5e-7a58-43af-ab73-9c7843099256-kube-api-access-2bhrr\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.492868 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.492757    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.589417 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.589382    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.689512 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.689469    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.790689 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.790587    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.890276 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.890230    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:40.913414 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.913383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-tls\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:40.916039 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:40.916012    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ebb15d5e-7a58-43af-ab73-9c7843099256-node-exporter-tls\") pod \"node-exporter-l2gjk\" (UID: \"ebb15d5e-7a58-43af-ab73-9c7843099256\") " pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:41.107881 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:41.107794    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l2gjk"
Jul 17 09:28:41.119376 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:41.119341    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb15d5e_7a58_43af_ab73_9c7843099256.slice/crio-d44e7010aad9e60bc587da3424d1188dfefc2933fe5b3e309b7df50fbab15e6e WatchSource:0}: Error finding container d44e7010aad9e60bc587da3424d1188dfefc2933fe5b3e309b7df50fbab15e6e: Status 404 returned error can't find the container with id d44e7010aad9e60bc587da3424d1188dfefc2933fe5b3e309b7df50fbab15e6e
Jul 17 09:28:41.159116 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:41.159078    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:41.238596 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:41.238563    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:41.359347 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:41.359260    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:42.085563 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:42.085526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l2gjk" event={"ID":"ebb15d5e-7a58-43af-ab73-9c7843099256","Type":"ContainerStarted","Data":"d44e7010aad9e60bc587da3424d1188dfefc2933fe5b3e309b7df50fbab15e6e"}
Jul 17 09:28:43.089916 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:43.089874    2556 generic.go:356] "Generic (PLEG): container finished" podID="ebb15d5e-7a58-43af-ab73-9c7843099256" containerID="2f9fe7b0f4212ea2a0ef6c1ef286fc3c7eb47f5a5bcc1984378567c5ae0d8828" exitCode=0
Jul 17 09:28:43.090279 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:43.089968    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l2gjk" event={"ID":"ebb15d5e-7a58-43af-ab73-9c7843099256","Type":"ContainerDied","Data":"2f9fe7b0f4212ea2a0ef6c1ef286fc3c7eb47f5a5bcc1984378567c5ae0d8828"}
Jul 17 09:28:43.177226 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:43.177193    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:43.272605 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:43.272574    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:43.376344 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:43.376312    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:44.095684 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.095650    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l2gjk" event={"ID":"ebb15d5e-7a58-43af-ab73-9c7843099256","Type":"ContainerStarted","Data":"5a768651223e798396950994636bdeb3cda2a0378fbb7075ad72107895906e30"}
Jul 17 09:28:44.096095 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.095708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l2gjk" event={"ID":"ebb15d5e-7a58-43af-ab73-9c7843099256","Type":"ContainerStarted","Data":"3b7e34680746ef65364c0f36e4b0c00fccee91f5e3a03b574c84989b9c9e2840"}
Jul 17 09:28:44.114374 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.114317    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l2gjk" podStartSLOduration=3.044789224 podStartE2EDuration="4.114299258s" totalImagesPullingTime="992.167013ms" totalInitContainerRuntime="77.343021ms" isStatefulPod=true podCreationTimestamp="2026-07-17 09:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:44.113115128 +0000 UTC m=+108.123239049" watchObservedRunningTime="2026-07-17 09:28:44.114299258 +0000 UTC m=+108.124423182"
Jul 17 09:28:44.975376 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.975341    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-856bc64749-spnh2"]
Jul 17 09:28:44.975572 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.975443    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:44.991768 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.991730    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"]
Jul 17 09:28:44.991968 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.991877    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:44.994417 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.994389    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 09:28:44.994997 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.994402    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 09:28:44.994997 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.994460    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 09:28:44.994997 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.994469    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-692fj\""
Jul 17 09:28:44.994997 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.994556    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 09:28:44.994997 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.994564    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 09:28:44.995639 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.995475    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 09:28:44.995639 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:44.995625    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 09:28:45.004999 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.004964    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 09:28:45.005568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.005538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"]
Jul 17 09:28:45.005692 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.005586    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-856bc64749-spnh2"]
Jul 17 09:28:45.005734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.005688    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.007939 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.007913    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 09:28:45.007939 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.007934    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j7plm\""
Jul 17 09:28:45.067366 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.067324    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:45.147149 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-serving-cert\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kf9x\" (UniqueName: \"kubernetes.io/projected/63a9ea49-2730-4eba-97cf-7c60358c3489-kube-api-access-9kf9x\") pod \"monitoring-plugin-7b7b7bc8bf-zbwzx\" (UID: \"63a9ea49-2730-4eba-97cf-7c60358c3489\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147275    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63a9ea49-2730-4eba-97cf-7c60358c3489-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zbwzx\" (UID: \"63a9ea49-2730-4eba-97cf-7c60358c3489\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147320    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-oauth-config\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147389    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-service-ca\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147411    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-trusted-ca-bundle\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147434    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-oauth-serving-cert\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147488    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-config\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.147568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.147541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d55n\" (UniqueName: \"kubernetes.io/projected/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-kube-api-access-9d55n\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.167124 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.167086    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:45.248296 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-config\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.248296 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248260    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d55n\" (UniqueName: \"kubernetes.io/projected/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-kube-api-access-9d55n\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.248577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248321    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-serving-cert\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.248577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248440    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kf9x\" (UniqueName: \"kubernetes.io/projected/63a9ea49-2730-4eba-97cf-7c60358c3489-kube-api-access-9kf9x\") pod \"monitoring-plugin-7b7b7bc8bf-zbwzx\" (UID: \"63a9ea49-2730-4eba-97cf-7c60358c3489\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.248709 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248607    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63a9ea49-2730-4eba-97cf-7c60358c3489-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zbwzx\" (UID: \"63a9ea49-2730-4eba-97cf-7c60358c3489\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.248709 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-oauth-config\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.248709 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248700    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-service-ca\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.248886 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248726    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-trusted-ca-bundle\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.248886 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.248772    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-oauth-serving-cert\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.249540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.249169    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-config\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.249540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.249419    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-service-ca\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.249738 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.249675    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-trusted-ca-bundle\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.249844 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.249802    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-oauth-serving-cert\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.251333 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.251307    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-serving-cert\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.251463 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.251374    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-oauth-config\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.251463 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.251439    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63a9ea49-2730-4eba-97cf-7c60358c3489-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-zbwzx\" (UID: \"63a9ea49-2730-4eba-97cf-7c60358c3489\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.255880 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.255852    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d55n\" (UniqueName: \"kubernetes.io/projected/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-kube-api-access-9d55n\") pod \"console-856bc64749-spnh2\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") " pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.255993 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.255858    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kf9x\" (UniqueName: \"kubernetes.io/projected/63a9ea49-2730-4eba-97cf-7c60358c3489-kube-api-access-9kf9x\") pod \"monitoring-plugin-7b7b7bc8bf-zbwzx\" (UID: \"63a9ea49-2730-4eba-97cf-7c60358c3489\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.267462 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.267429    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:45.308693 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.308643    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:45.317811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.317776    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:45.369498 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.369454    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:45.475562 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.475526    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-856bc64749-spnh2"]
Jul 17 09:28:45.478910 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:45.478880    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b7a6bf9_8a99_44ee_ac22_5c9978497ebe.slice/crio-f98e0a8d50984f36fd46ead91201847015964423bace5227c3ef292a0b87153c WatchSource:0}: Error finding container f98e0a8d50984f36fd46ead91201847015964423bace5227c3ef292a0b87153c: Status 404 returned error can't find the container with id f98e0a8d50984f36fd46ead91201847015964423bace5227c3ef292a0b87153c
Jul 17 09:28:45.503774 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:45.503691    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"]
Jul 17 09:28:45.506899 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:28:45.506871    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a9ea49_2730_4eba_97cf_7c60358c3489.slice/crio-40c5554b5621673d58ddaeda6b7dcc0497d1f406fafee8c510d4bc560aa57793 WatchSource:0}: Error finding container 40c5554b5621673d58ddaeda6b7dcc0497d1f406fafee8c510d4bc560aa57793: Status 404 returned error can't find the container with id 40c5554b5621673d58ddaeda6b7dcc0497d1f406fafee8c510d4bc560aa57793
Jul 17 09:28:46.104922 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:46.104879    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx" event={"ID":"63a9ea49-2730-4eba-97cf-7c60358c3489","Type":"ContainerStarted","Data":"40c5554b5621673d58ddaeda6b7dcc0497d1f406fafee8c510d4bc560aa57793"}
Jul 17 09:28:46.106974 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:46.106936    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-856bc64749-spnh2" event={"ID":"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe","Type":"ContainerStarted","Data":"f98e0a8d50984f36fd46ead91201847015964423bace5227c3ef292a0b87153c"}
Jul 17 09:28:46.431132 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:46.431095    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:46.936615 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:46.936526    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:46.987627 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:46.987595    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:47.320194 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:47.320090    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:47.422690 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:47.422654    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:49.121228 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:49.121196    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx" event={"ID":"63a9ea49-2730-4eba-97cf-7c60358c3489","Type":"ContainerStarted","Data":"a74c5b187e93aa9cd465b77b8f399a613d5a037421c4a5d8f52dfbaf59fb53de"}
Jul 17 09:28:49.121660 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:49.121306    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:49.126937 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:49.126896    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx"
Jul 17 09:28:49.137534 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:49.137458    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-zbwzx" podStartSLOduration=1.961843758 podStartE2EDuration="5.137440731s" totalImagesPullingTime="3.175596973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:49.136547657 +0000 UTC m=+113.146671583" watchObservedRunningTime="2026-07-17 09:28:49.137440731 +0000 UTC m=+113.147564654"
Jul 17 09:28:50.128781 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:50.128738    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-856bc64749-spnh2" event={"ID":"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe","Type":"ContainerStarted","Data":"96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7"}
Jul 17 09:28:50.146151 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:50.146095    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-856bc64749-spnh2" podStartSLOduration=2.630429776 podStartE2EDuration="6.146073489s" totalImagesPullingTime="3.515643713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:50.144655931 +0000 UTC m=+114.154779854" watchObservedRunningTime="2026-07-17 09:28:50.146073489 +0000 UTC m=+114.156197412"
Jul 17 09:28:51.340177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:51.340137    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-856bc64749-spnh2"]
Jul 17 09:28:51.493429 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:51.493393    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:51.593261 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:51.593161    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:54.171521 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:54.171476    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:54.272046 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:54.272009    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:55.309596 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:55.309557    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:28:55.586189 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:55.586106    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:28:55.644202 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:28:55.644153    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:01.921445 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:01.921410    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:01.984664 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:01.984627    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:02.674791 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:02.674759    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:02.731523 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:02.731486    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:02.782612 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:02.782578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:17.156617 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.156568    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-856bc64749-spnh2" podUID="3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" containerName="console" containerID="cri-o://96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7" gracePeriod=15
Jul 17 09:29:17.410862 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.410775    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:29:17.494766 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.494725    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-oauth-config\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.494971 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.494782    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-serving-cert\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.494971 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.494799    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-oauth-serving-cert\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.494971 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.494841    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-config\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.495130 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495044    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-trusted-ca-bundle\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.495183 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495125    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-service-ca\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.495183 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495159    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d55n\" (UniqueName: \"kubernetes.io/projected/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-kube-api-access-9d55n\") pod \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\" (UID: \"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe\") "
Jul 17 09:29:17.495310 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495285    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-config" (OuterVolumeSpecName: "console-config") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:17.495398 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495382    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-config\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:17.495449 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495337    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:17.495494 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495431    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:17.495657 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.495519    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-service-ca" (OuterVolumeSpecName: "service-ca") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:17.497353 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.497323    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-kube-api-access-9d55n" (OuterVolumeSpecName: "kube-api-access-9d55n") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "kube-api-access-9d55n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:29:17.497353 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.497333    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:29:17.497484 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.497356    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" (UID: "3b7a6bf9-8a99-44ee-ac22-5c9978497ebe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:29:17.596716 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.596670    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-serving-cert\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:17.596716 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.596710    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-oauth-serving-cert\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:17.596716 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.596725    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-trusted-ca-bundle\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:17.596993 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.596737    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:17.596993 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.596749    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d55n\" (UniqueName: \"kubernetes.io/projected/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-kube-api-access-9d55n\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:17.596993 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:17.596760    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe-console-oauth-config\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:29:18.232063 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.232021    2556 generic.go:356] "Generic (PLEG): container finished" podID="3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" containerID="96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7" exitCode=0
Jul 17 09:29:18.232565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.232080    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-856bc64749-spnh2"
Jul 17 09:29:18.232565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.232105    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-856bc64749-spnh2" event={"ID":"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe","Type":"ContainerDied","Data":"96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7"}
Jul 17 09:29:18.232565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.232162    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-856bc64749-spnh2" event={"ID":"3b7a6bf9-8a99-44ee-ac22-5c9978497ebe","Type":"ContainerDied","Data":"f98e0a8d50984f36fd46ead91201847015964423bace5227c3ef292a0b87153c"}
Jul 17 09:29:18.232565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.232197    2556 scope.go:117] "RemoveContainer" containerID="96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7"
Jul 17 09:29:18.241168 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.241145    2556 scope.go:117] "RemoveContainer" containerID="96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7"
Jul 17 09:29:18.241544 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:29:18.241522    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7\": container with ID starting with 96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7 not found: ID does not exist" containerID="96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7"
Jul 17 09:29:18.241608 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.241556    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7"} err="failed to get container status \"96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7\": rpc error: code = NotFound desc = could not find container \"96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7\": container with ID starting with 96e9785cc0e97b6b1f4a331088cf3b53dbcab0cdfd48622f0483c75ef41dade7 not found: ID does not exist"
Jul 17 09:29:18.253153 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.253118    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-856bc64749-spnh2"]
Jul 17 09:29:18.256772 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.256747    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-856bc64749-spnh2"]
Jul 17 09:29:18.494252 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:18.494171    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" path="/var/lib/kubelet/pods/3b7a6bf9-8a99-44ee-ac22-5c9978497ebe/volumes"
Jul 17 09:29:56.358327 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:56.358292    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:29:56.416277 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:29:56.416217    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:00.166182 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.166147    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b8bbf4755-r97sq"]
Jul 17 09:30:00.166595 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.166395    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" containerName="console"
Jul 17 09:30:00.166595 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.166406    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" containerName="console"
Jul 17 09:30:00.166595 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.166455    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b7a6bf9-8a99-44ee-ac22-5c9978497ebe" containerName="console"
Jul 17 09:30:00.170565 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.170543    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.174907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174729    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 09:30:00.174907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174776    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 09:30:00.174907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174792    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 09:30:00.174907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174813    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 09:30:00.174907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174886    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-692fj\""
Jul 17 09:30:00.174907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174898    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 09:30:00.175328 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.174889    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 09:30:00.175519 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.175502    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 09:30:00.181089 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.180673    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 09:30:00.183866 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.183815    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8bbf4755-r97sq"]
Jul 17 09:30:00.224394 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.224352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:00.289017 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.288980    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:00.321911 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.321878    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-oauth-serving-cert\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.322015 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.321919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-oauth-config\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.322015 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.321940    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-trusted-ca-bundle\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.322015 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.321972    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-service-ca\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.322015 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.322006    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-serving-cert\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.322169 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.322036    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwjb\" (UniqueName: \"kubernetes.io/projected/229aee67-9b79-46a5-a799-b47f040541d1-kube-api-access-bbwjb\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.322169 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.322077    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-console-config\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.354278 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.354247    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:00.423492 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-console-config\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.423492 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423448    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-oauth-serving-cert\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.423740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-oauth-config\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.423740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423625    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-trusted-ca-bundle\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.423740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423663    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-service-ca\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.423740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-serving-cert\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.423740 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.423740    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwjb\" (UniqueName: \"kubernetes.io/projected/229aee67-9b79-46a5-a799-b47f040541d1-kube-api-access-bbwjb\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.424302 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.424269    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-console-config\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.424426 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.424268    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-oauth-serving-cert\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.424561 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.424538    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-service-ca\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.424714 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.424693    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-trusted-ca-bundle\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.426206 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.426179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-oauth-config\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.426293 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.426275    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-serving-cert\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.432321 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.432289    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwjb\" (UniqueName: \"kubernetes.io/projected/229aee67-9b79-46a5-a799-b47f040541d1-kube-api-access-bbwjb\") pod \"console-6b8bbf4755-r97sq\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") " pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.480772 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.480734    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:00.619634 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:00.619589    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b8bbf4755-r97sq"]
Jul 17 09:30:00.623581 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:30:00.623556    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229aee67_9b79_46a5_a799_b47f040541d1.slice/crio-716583b09430eb071d7efd819222102e188a47752f9b3be630331f850cdae388 WatchSource:0}: Error finding container 716583b09430eb071d7efd819222102e188a47752f9b3be630331f850cdae388: Status 404 returned error can't find the container with id 716583b09430eb071d7efd819222102e188a47752f9b3be630331f850cdae388
Jul 17 09:30:01.173409 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:01.173367    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:01.241450 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:01.241412    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:01.386681 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:01.386643    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bbf4755-r97sq" event={"ID":"229aee67-9b79-46a5-a799-b47f040541d1","Type":"ContainerStarted","Data":"315f5a690cd2e70ab7af03d16a169f2870f41506d24d05a342f54b56fec65f3b"}
Jul 17 09:30:01.386913 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:01.386699    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bbf4755-r97sq" event={"ID":"229aee67-9b79-46a5-a799-b47f040541d1","Type":"ContainerStarted","Data":"716583b09430eb071d7efd819222102e188a47752f9b3be630331f850cdae388"}
Jul 17 09:30:01.404058 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:01.404006    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b8bbf4755-r97sq" podStartSLOduration=1.403988949 podStartE2EDuration="1.403988949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:30:01.403091347 +0000 UTC m=+185.413215272" watchObservedRunningTime="2026-07-17 09:30:01.403988949 +0000 UTC m=+185.414112872"
Jul 17 09:30:08.208557 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:08.208505    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:08.309096 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:08.309057    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:10.481386 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:10.481341    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:10.481816 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:10.481430    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:10.486510 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:10.486479    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:11.427228 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:11.427198    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:30:11.571528 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:11.571492    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:11.672137 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:11.672098    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:11.726304 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:11.726213    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:16.940404 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:16.940364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:17.041249 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:17.041207    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:26.641224 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:26.641188    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:26.696840 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:26.696783    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:35.500299 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:35.500265    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:47.322406 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:47.322365    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:49.785231 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:49.785192    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:49.882194 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:49.882155    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:30:52.496190 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:30:52.496143    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:11.399002 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:11.398969    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:12.186472 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:12.186432    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:13.276892 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.276850    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"]
Jul 17 09:31:13.280215 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.280187    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.282687 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.282654    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:31:13.282887 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.282690    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 17 09:31:13.282887 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.282755    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-6pzk9\""
Jul 17 09:31:13.292583 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.292555    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"]
Jul 17 09:31:13.351475 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.351432    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1d73245-1ce8-4a6f-841d-4daecdd7074c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lddhq\" (UID: \"a1d73245-1ce8-4a6f-841d-4daecdd7074c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.351475 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.351480    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xzpt\" (UniqueName: \"kubernetes.io/projected/a1d73245-1ce8-4a6f-841d-4daecdd7074c-kube-api-access-7xzpt\") pod \"cert-manager-operator-controller-manager-55c444bf86-lddhq\" (UID: \"a1d73245-1ce8-4a6f-841d-4daecdd7074c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.452773 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.452733    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1d73245-1ce8-4a6f-841d-4daecdd7074c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lddhq\" (UID: \"a1d73245-1ce8-4a6f-841d-4daecdd7074c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.452773 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.452777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xzpt\" (UniqueName: \"kubernetes.io/projected/a1d73245-1ce8-4a6f-841d-4daecdd7074c-kube-api-access-7xzpt\") pod \"cert-manager-operator-controller-manager-55c444bf86-lddhq\" (UID: \"a1d73245-1ce8-4a6f-841d-4daecdd7074c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.453205 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.453181    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1d73245-1ce8-4a6f-841d-4daecdd7074c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lddhq\" (UID: \"a1d73245-1ce8-4a6f-841d-4daecdd7074c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.460928 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.460895    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xzpt\" (UniqueName: \"kubernetes.io/projected/a1d73245-1ce8-4a6f-841d-4daecdd7074c-kube-api-access-7xzpt\") pod \"cert-manager-operator-controller-manager-55c444bf86-lddhq\" (UID: \"a1d73245-1ce8-4a6f-841d-4daecdd7074c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.589953 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.589852    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"
Jul 17 09:31:13.736171 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:13.736035    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq"]
Jul 17 09:31:13.739410 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:31:13.739378    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d73245_1ce8_4a6f_841d_4daecdd7074c.slice/crio-62ab2bfefcf247d4aaf065793e52e59dd119eaf04bda4be3f4c06bf51143f302 WatchSource:0}: Error finding container 62ab2bfefcf247d4aaf065793e52e59dd119eaf04bda4be3f4c06bf51143f302: Status 404 returned error can't find the container with id 62ab2bfefcf247d4aaf065793e52e59dd119eaf04bda4be3f4c06bf51143f302
Jul 17 09:31:14.644366 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:14.644303    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq" event={"ID":"a1d73245-1ce8-4a6f-841d-4daecdd7074c","Type":"ContainerStarted","Data":"62ab2bfefcf247d4aaf065793e52e59dd119eaf04bda4be3f4c06bf51143f302"}
Jul 17 09:31:14.983229 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:14.983135    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:15.082012 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:15.081962    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:16.654742 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:16.654655    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq" event={"ID":"a1d73245-1ce8-4a6f-841d-4daecdd7074c","Type":"ContainerStarted","Data":"5940bbeca16fa81937c960bbbb39b8157cf2145fd20793a7a83363d9ffb70829"}
Jul 17 09:31:16.680495 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:16.680357    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lddhq" podStartSLOduration=1.00466953 podStartE2EDuration="3.680339705s" totalImagesPullingTime="2.675670175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:31:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:31:16.678650006 +0000 UTC m=+260.688773932" watchObservedRunningTime="2026-07-17 09:31:16.680339705 +0000 UTC m=+260.690463628"
Jul 17 09:31:17.091681 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:17.091604    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:17.263695 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:17.263651    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:17.394878 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:17.394814    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:17.495367 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:17.495333    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:19.686276 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:19.686241    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:19.882177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:19.882144    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:21.769168 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:21.769132    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:21.872304 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:21.872268    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:27.291461 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:27.291422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:29.455929 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:29.455889    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:29.555309 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:29.555270    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:29.667640 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:29.667606    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:29.722352 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:29.722268    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:34.816069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:34.816033    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:34.874746 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:34.874707    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:36.776131 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:36.776041    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:50.401325 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:50.401290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:50.500715 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:50.500681    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.113343 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.113303    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"]
Jul 17 09:31:51.115568 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.115548    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.118348 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.118324    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 09:31:51.119538 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.119504    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 17 09:31:51.119538 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.119532    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hcnr9\""
Jul 17 09:31:51.119732 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.119565    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:31:51.119732 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.119581    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 17 09:31:51.119732 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.119540    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 17 09:31:51.128325 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.128298    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"]
Jul 17 09:31:51.133234 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.133202    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7kn\" (UniqueName: \"kubernetes.io/projected/597954db-f1e2-4cea-b22e-40826bd0fb86-kube-api-access-9g7kn\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.133234 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.133244    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/597954db-f1e2-4cea-b22e-40826bd0fb86-metrics-cert\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.133435 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.133281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/597954db-f1e2-4cea-b22e-40826bd0fb86-manager-config\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.133435 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.133377    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/597954db-f1e2-4cea-b22e-40826bd0fb86-cert\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.141977 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.141947    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.233805 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.233757    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/597954db-f1e2-4cea-b22e-40826bd0fb86-cert\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.234042 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.233856    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7kn\" (UniqueName: \"kubernetes.io/projected/597954db-f1e2-4cea-b22e-40826bd0fb86-kube-api-access-9g7kn\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.234042 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.233886    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/597954db-f1e2-4cea-b22e-40826bd0fb86-metrics-cert\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.234042 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.233934    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/597954db-f1e2-4cea-b22e-40826bd0fb86-manager-config\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.234673 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.234644    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/597954db-f1e2-4cea-b22e-40826bd0fb86-manager-config\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.236613 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.236589    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/597954db-f1e2-4cea-b22e-40826bd0fb86-cert\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.236742 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.236632    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/597954db-f1e2-4cea-b22e-40826bd0fb86-metrics-cert\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.248341 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.248310    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7kn\" (UniqueName: \"kubernetes.io/projected/597954db-f1e2-4cea-b22e-40826bd0fb86-kube-api-access-9g7kn\") pod \"lws-controller-manager-7df87f8d9d-mdwn6\" (UID: \"597954db-f1e2-4cea-b22e-40826bd0fb86\") " pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.277754 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.277716    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.376859 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.376803    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.424617 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.424588    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:51.483804 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.483753    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.518273 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.518236    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"]
Jul 17 09:31:51.521267 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.521241    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.524769 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.524742    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7g64w\""
Jul 17 09:31:51.525167 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.525142    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 09:31:51.525459 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.525440    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 09:31:51.525558 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.525448    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 09:31:51.526045 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.526027    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 09:31:51.536430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.536126    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fd30b3c-b529-4a08-a003-a63d1f4a002a-manifests\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.536430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.536226    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fd30b3c-b529-4a08-a003-a63d1f4a002a-tmp\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.536430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.536282    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fd30b3c-b529-4a08-a003-a63d1f4a002a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.536430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.536307    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fd30b3c-b529-4a08-a003-a63d1f4a002a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.536430 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.536348    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ng6m\" (UniqueName: \"kubernetes.io/projected/7fd30b3c-b529-4a08-a003-a63d1f4a002a-kube-api-access-4ng6m\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.551556 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.551521    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"]
Jul 17 09:31:51.581004 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.580979    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"]
Jul 17 09:31:51.582577 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:31:51.582554    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597954db_f1e2_4cea_b22e_40826bd0fb86.slice/crio-2cbdcabd25c393d915014e39ef01eaee752301f91f8ab83241c89869277264a6 WatchSource:0}: Error finding container 2cbdcabd25c393d915014e39ef01eaee752301f91f8ab83241c89869277264a6: Status 404 returned error can't find the container with id 2cbdcabd25c393d915014e39ef01eaee752301f91f8ab83241c89869277264a6
Jul 17 09:31:51.637536 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.637435    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fd30b3c-b529-4a08-a003-a63d1f4a002a-manifests\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.637536 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.637495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fd30b3c-b529-4a08-a003-a63d1f4a002a-tmp\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.637536 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.637528    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fd30b3c-b529-4a08-a003-a63d1f4a002a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.637811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.637548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fd30b3c-b529-4a08-a003-a63d1f4a002a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.637811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.637575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ng6m\" (UniqueName: \"kubernetes.io/projected/7fd30b3c-b529-4a08-a003-a63d1f4a002a-kube-api-access-4ng6m\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.637960 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.637896    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7fd30b3c-b529-4a08-a003-a63d1f4a002a-manifests\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.640118 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.640090    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7fd30b3c-b529-4a08-a003-a63d1f4a002a-tmp\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.640325 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.640304    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fd30b3c-b529-4a08-a003-a63d1f4a002a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.640373 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.640304    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7fd30b3c-b529-4a08-a003-a63d1f4a002a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.653465 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.653432    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ng6m\" (UniqueName: \"kubernetes.io/projected/7fd30b3c-b529-4a08-a003-a63d1f4a002a-kube-api-access-4ng6m\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-2q7sh\" (UID: \"7fd30b3c-b529-4a08-a003-a63d1f4a002a\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.717017 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.716973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.795549 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.795516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6" event={"ID":"597954db-f1e2-4cea-b22e-40826bd0fb86","Type":"ContainerStarted","Data":"2cbdcabd25c393d915014e39ef01eaee752301f91f8ab83241c89869277264a6"}
Jul 17 09:31:51.811266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.811229    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.834374 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.834338    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:51.912540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.912447    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.988644 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.988605    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"]
Jul 17 09:31:51.991493 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:51.991464    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:51.992413 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:31:51.992390    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd30b3c_b529_4a08_a003_a63d1f4a002a.slice/crio-349dd03107a243dbf87af5d2864b28ad2386926c46e107342b37dbb76e7551c0 WatchSource:0}: Error finding container 349dd03107a243dbf87af5d2864b28ad2386926c46e107342b37dbb76e7551c0: Status 404 returned error can't find the container with id 349dd03107a243dbf87af5d2864b28ad2386926c46e107342b37dbb76e7551c0
Jul 17 09:31:52.070315 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:52.070279    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:52.184021 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:52.183925    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:31:52.804392 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:52.804350    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh" event={"ID":"7fd30b3c-b529-4a08-a003-a63d1f4a002a","Type":"ContainerStarted","Data":"349dd03107a243dbf87af5d2864b28ad2386926c46e107342b37dbb76e7551c0"}
Jul 17 09:31:56.366715 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:56.366685    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 09:31:56.823125 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:56.823087    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh" event={"ID":"7fd30b3c-b529-4a08-a003-a63d1f4a002a","Type":"ContainerStarted","Data":"c45b1ec9459904983181bd229c33bf58ad70977f0886011f2f054e1a11206847"}
Jul 17 09:31:56.826157 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:56.824698    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6" event={"ID":"597954db-f1e2-4cea-b22e-40826bd0fb86","Type":"ContainerStarted","Data":"b3760ce6036c1ab39ff39dfcf3c8eaab874fb272c0947c588c98bc26d877bdf4"}
Jul 17 09:31:56.826157 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:56.824868    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:31:56.881514 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:56.881459    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6" podStartSLOduration=1.280705239 podStartE2EDuration="5.881442287s" totalImagesPullingTime="4.600737048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:31:56.880437204 +0000 UTC m=+300.890561129" watchObservedRunningTime="2026-07-17 09:31:56.881442287 +0000 UTC m=+300.891566212"
Jul 17 09:31:57.830527 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:57.830415    2556 generic.go:356] "Generic (PLEG): container finished" podID="7fd30b3c-b529-4a08-a003-a63d1f4a002a" containerID="c45b1ec9459904983181bd229c33bf58ad70977f0886011f2f054e1a11206847" exitCode=0
Jul 17 09:31:57.831026 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:57.830520    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh" event={"ID":"7fd30b3c-b529-4a08-a003-a63d1f4a002a","Type":"ContainerDied","Data":"c45b1ec9459904983181bd229c33bf58ad70977f0886011f2f054e1a11206847"}
Jul 17 09:31:57.831075 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:57.831025    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:31:58.836379 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:58.836336    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh" event={"ID":"7fd30b3c-b529-4a08-a003-a63d1f4a002a","Type":"ContainerStarted","Data":"f3590f59ffe5f06e5e7ac882d483bd216753683538ee7e75d72a2bb210e504e0"}
Jul 17 09:31:58.836907 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:58.836416    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:31:58.857322 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:31:58.857256    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh" podStartSLOduration=2.189523982 podStartE2EDuration="7.857234424s" totalImagesPullingTime="4.538660443s" totalInitContainerRuntime="1.129049999s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:31:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:31:58.855815306 +0000 UTC m=+302.865939229" watchObservedRunningTime="2026-07-17 09:31:58.857234424 +0000 UTC m=+302.867358348"
Jul 17 09:32:02.991074 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:02.991026    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:04.454621 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:04.454585    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:05.236751 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:05.236712    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:05.337491 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:05.337455    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:07.833332 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:07.833292    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7df87f8d9d-mdwn6"
Jul 17 09:32:09.842741 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:09.842703    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-2q7sh"
Jul 17 09:32:11.454804 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.454741    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:11.564135 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.564100    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:11.918178 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.918140    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:11.954689 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.954656    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"]
Jul 17 09:32:11.974509 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.974472    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"]
Jul 17 09:32:11.974692 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.974626    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:11.977338 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.977305    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jddfn\""
Jul 17 09:32:11.977655 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.977636    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 09:32:11.978084 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:11.978067    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 09:32:12.113755 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.113710    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj2cx\" (UniqueName: \"kubernetes.io/projected/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-kube-api-access-gj2cx\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.113958 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.113771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-tls-certs\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.113958 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.113787    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-tmp\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.208127 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.208035    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:12.215190 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.215157    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj2cx\" (UniqueName: \"kubernetes.io/projected/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-kube-api-access-gj2cx\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.215366 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.215214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-tls-certs\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.215366 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.215237    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-tmp\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.218002 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.217961    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-tmp\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.218196 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.218179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-tls-certs\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.226896 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.226865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj2cx\" (UniqueName: \"kubernetes.io/projected/b5fb7163-04a7-4c5a-b251-e0c4bd52f927-kube-api-access-gj2cx\") pod \"kube-auth-proxy-546bc49978-jhjq6\" (UID: \"b5fb7163-04a7-4c5a-b251-e0c4bd52f927\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.285780 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.285737    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"
Jul 17 09:32:12.301203 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.301163    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:12.402857 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.402808    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:12.425776 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.425726    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-546bc49978-jhjq6"]
Jul 17 09:32:12.428775 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:32:12.428733    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5fb7163_04a7_4c5a_b251_e0c4bd52f927.slice/crio-2a866c9ec4100360657017c47ec8fb57dc159bf8fa87489cf4552b108d5bf1aa WatchSource:0}: Error finding container 2a866c9ec4100360657017c47ec8fb57dc159bf8fa87489cf4552b108d5bf1aa: Status 404 returned error can't find the container with id 2a866c9ec4100360657017c47ec8fb57dc159bf8fa87489cf4552b108d5bf1aa
Jul 17 09:32:12.896194 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:12.896157    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6" event={"ID":"b5fb7163-04a7-4c5a-b251-e0c4bd52f927","Type":"ContainerStarted","Data":"2a866c9ec4100360657017c47ec8fb57dc159bf8fa87489cf4552b108d5bf1aa"}
Jul 17 09:32:13.007217 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:13.007175    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:13.270440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:13.270336    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:13.342131 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:13.342084    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:13.691340 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:13.691273    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:14.197872 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:14.197811    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:15.910964 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:15.910924    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6" event={"ID":"b5fb7163-04a7-4c5a-b251-e0c4bd52f927","Type":"ContainerStarted","Data":"0b8777478a24b5e4d7d91a392cb0a2982931dd610b10dba52c3bbb1328479a37"}
Jul 17 09:32:15.926783 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:15.926722    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-546bc49978-jhjq6" podStartSLOduration=1.7593190920000001 podStartE2EDuration="4.926705812s" totalImagesPullingTime="3.16738672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:15.924653355 +0000 UTC m=+319.934777304" watchObservedRunningTime="2026-07-17 09:32:15.926705812 +0000 UTC m=+319.936829737"
Jul 17 09:32:19.228373 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:19.228323    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:19.296435 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:19.296392    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:19.857022 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:19.856960    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:20.240781 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:20.240748    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:20.352961 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:20.352926    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:23.887218 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:23.887169    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:23.984496 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:23.984439    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:24.299765 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:24.299641    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:25.102998 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:25.102965    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:25.196055 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:25.195987    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:41.985781 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:41.985735    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:51.489887 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:51.489757    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:52.337209 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:52.337163    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:32:52.392962 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:32:52.392928    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:12.926553 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:12.926516    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:13.030559 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:13.030519    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:13.131151 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:13.131111    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:13.866117 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:13.866064    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:14.118481 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:14.118385    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:42.249025 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:42.248982    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:43.059865 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:43.059802    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:46.097093 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:46.097044    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:47.447289 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:47.447251    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:47.871182 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:47.871128    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:48.901217 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:48.901178    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:50.462242 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:50.462180    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:50.562153 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:50.562111    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:52.066024 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:52.065991    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:52.175582 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:52.175547    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:52.278197 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:52.278149    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:52.678133 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:52.678093    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:52.785236 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:52.785207    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:52.910948 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:52.910896    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:54.528089 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:54.528052    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:56.259527 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:56.259487    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:56.359905 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:56.359854    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:57.225729 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.225685    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"]
Jul 17 09:33:57.227973 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.227952    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.231302 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.231279    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 09:33:57.231446 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.231366    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k2wpb\""
Jul 17 09:33:57.231446 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.231405    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 17 09:33:57.231595 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.231468    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 09:33:57.232041 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.232024    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 17 09:33:57.237750 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.237723    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"]
Jul 17 09:33:57.240802 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.240772    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:57.272245 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.272204    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e23b0300-48e8-4515-acc2-79019874acea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.272245 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.272246    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkwrh\" (UniqueName: \"kubernetes.io/projected/e23b0300-48e8-4515-acc2-79019874acea-kube-api-access-lkwrh\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.272734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.272280    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e23b0300-48e8-4515-acc2-79019874acea-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.373160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.373119    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkwrh\" (UniqueName: \"kubernetes.io/projected/e23b0300-48e8-4515-acc2-79019874acea-kube-api-access-lkwrh\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.373377 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.373178    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e23b0300-48e8-4515-acc2-79019874acea-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.373377 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.373245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e23b0300-48e8-4515-acc2-79019874acea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.373377 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:33:57.373347    2556 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 17 09:33:57.373542 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:33:57.373407    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e23b0300-48e8-4515-acc2-79019874acea-plugin-serving-cert podName:e23b0300-48e8-4515-acc2-79019874acea nodeName:}" failed. No retries permitted until 2026-07-17 09:33:57.873387656 +0000 UTC m=+421.883511560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e23b0300-48e8-4515-acc2-79019874acea-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-mvqhv" (UID: "e23b0300-48e8-4515-acc2-79019874acea") : secret "plugin-serving-cert" not found
Jul 17 09:33:57.373942 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.373920    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e23b0300-48e8-4515-acc2-79019874acea-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.382261 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.382232    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkwrh\" (UniqueName: \"kubernetes.io/projected/e23b0300-48e8-4515-acc2-79019874acea-kube-api-access-lkwrh\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.580491 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.580397    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:33:57.876268 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.876212    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e23b0300-48e8-4515-acc2-79019874acea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:57.878974 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:57.878950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e23b0300-48e8-4515-acc2-79019874acea-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mvqhv\" (UID: \"e23b0300-48e8-4515-acc2-79019874acea\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:58.139160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:58.139062    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"
Jul 17 09:33:58.289012 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:58.288977    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv"]
Jul 17 09:33:58.292186 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:33:58.292143    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode23b0300_48e8_4515_acc2_79019874acea.slice/crio-7ff6ddfbe5775764779d6f233ac7e84e988310f481d7853c65b5984851aad325 WatchSource:0}: Error finding container 7ff6ddfbe5775764779d6f233ac7e84e988310f481d7853c65b5984851aad325: Status 404 returned error can't find the container with id 7ff6ddfbe5775764779d6f233ac7e84e988310f481d7853c65b5984851aad325
Jul 17 09:33:58.333170 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:58.333130    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv" event={"ID":"e23b0300-48e8-4515-acc2-79019874acea","Type":"ContainerStarted","Data":"7ff6ddfbe5775764779d6f233ac7e84e988310f481d7853c65b5984851aad325"}
Jul 17 09:33:59.950957 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:33:59.950914    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:00.015686 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:00.015650    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:02.909809 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:02.909397    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8bbf4755-r97sq"]
Jul 17 09:34:03.032468 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:03.032376    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:03.130520 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:03.130474    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:03.229482 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:03.229395    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:05.470722 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:05.470689    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:05.569676 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:05.569621    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:07.983654 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:07.983534    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:08.090534 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:08.090492    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:12.670888 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:12.670847    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:12.790102 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:12.790061    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:16.472267 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:16.472226    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:16.530328 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:16.530287    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:21.457273 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:21.457236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv" event={"ID":"e23b0300-48e8-4515-acc2-79019874acea","Type":"ContainerStarted","Data":"7030e8febe92dc6423a57f49a4ca32a439b2de6e41cc11e7ae49ade7d711fc57"}
Jul 17 09:34:21.476181 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:21.476126    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mvqhv" podStartSLOduration=2.33740622 podStartE2EDuration="24.476108174s" totalImagesPullingTime="22.138701954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:21.475521672 +0000 UTC m=+445.485645592" watchObservedRunningTime="2026-07-17 09:34:21.476108174 +0000 UTC m=+445.486232110"
Jul 17 09:34:25.167613 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:25.167571    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:25.267294 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:25.267257    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:27.608792 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:27.608745    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:27.661564 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:27.661529    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:27.943999 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:27.943959    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b8bbf4755-r97sq" podUID="229aee67-9b79-46a5-a799-b47f040541d1" containerName="console" containerID="cri-o://315f5a690cd2e70ab7af03d16a169f2870f41506d24d05a342f54b56fec65f3b" gracePeriod=15
Jul 17 09:34:28.489760 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.489723    2556 generic.go:356] "Generic (PLEG): container finished" podID="229aee67-9b79-46a5-a799-b47f040541d1" containerID="315f5a690cd2e70ab7af03d16a169f2870f41506d24d05a342f54b56fec65f3b" exitCode=0
Jul 17 09:34:28.494099 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.494066    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bbf4755-r97sq" event={"ID":"229aee67-9b79-46a5-a799-b47f040541d1","Type":"ContainerDied","Data":"315f5a690cd2e70ab7af03d16a169f2870f41506d24d05a342f54b56fec65f3b"}
Jul 17 09:34:28.789165 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.789119    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:34:28.867456 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867419    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-console-config\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867456 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867467    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-service-ca\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867704 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867493    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbwjb\" (UniqueName: \"kubernetes.io/projected/229aee67-9b79-46a5-a799-b47f040541d1-kube-api-access-bbwjb\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867704 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867517    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-oauth-config\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867704 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867557    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-oauth-serving-cert\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867704 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867596    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-trusted-ca-bundle\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867704 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867680    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-serving-cert\") pod \"229aee67-9b79-46a5-a799-b47f040541d1\" (UID: \"229aee67-9b79-46a5-a799-b47f040541d1\") "
Jul 17 09:34:28.867977 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867930    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-console-config" (OuterVolumeSpecName: "console-config") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:34:28.868042 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.867915    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-service-ca" (OuterVolumeSpecName: "service-ca") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:34:28.868113 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.868092    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:34:28.868160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.868133    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:34:28.870112 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.870082    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:34:28.870112 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.870077    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:34:28.870262 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.870085    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/229aee67-9b79-46a5-a799-b47f040541d1-kube-api-access-bbwjb" (OuterVolumeSpecName: "kube-api-access-bbwjb") pod "229aee67-9b79-46a5-a799-b47f040541d1" (UID: "229aee67-9b79-46a5-a799-b47f040541d1"). InnerVolumeSpecName "kube-api-access-bbwjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:34:28.968657 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968611    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-serving-cert\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:28.968657 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968648    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-console-config\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:28.968657 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968657    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:28.968657 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968666    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbwjb\" (UniqueName: \"kubernetes.io/projected/229aee67-9b79-46a5-a799-b47f040541d1-kube-api-access-bbwjb\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:28.968962 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968676    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/229aee67-9b79-46a5-a799-b47f040541d1-console-oauth-config\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:28.968962 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968684    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-oauth-serving-cert\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:28.968962 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:28.968693    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/229aee67-9b79-46a5-a799-b47f040541d1-trusted-ca-bundle\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:29.494882 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:29.494834    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b8bbf4755-r97sq"
Jul 17 09:34:29.495064 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:29.494843    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b8bbf4755-r97sq" event={"ID":"229aee67-9b79-46a5-a799-b47f040541d1","Type":"ContainerDied","Data":"716583b09430eb071d7efd819222102e188a47752f9b3be630331f850cdae388"}
Jul 17 09:34:29.495064 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:29.494978    2556 scope.go:117] "RemoveContainer" containerID="315f5a690cd2e70ab7af03d16a169f2870f41506d24d05a342f54b56fec65f3b"
Jul 17 09:34:29.514850 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:29.514804    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b8bbf4755-r97sq"]
Jul 17 09:34:29.518694 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:29.518660    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b8bbf4755-r97sq"]
Jul 17 09:34:30.495180 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:30.495141    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="229aee67-9b79-46a5-a799-b47f040541d1" path="/var/lib/kubelet/pods/229aee67-9b79-46a5-a799-b47f040541d1/volumes"
Jul 17 09:34:32.373978 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:32.373942    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:32.479553 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:32.479523    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:36.378946 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:36.378902    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:36.508324 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:36.508283    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:37.314946 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:37.314913    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:37.540090 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:37.540051    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:42.096011 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.095967    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:42.212951 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.212915    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:42.324472 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.324420    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:42.511659 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.511620    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:34:42.511933 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.511920    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="229aee67-9b79-46a5-a799-b47f040541d1" containerName="console"
Jul 17 09:34:42.511991 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.511935    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="229aee67-9b79-46a5-a799-b47f040541d1" containerName="console"
Jul 17 09:34:42.512026 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.511995    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="229aee67-9b79-46a5-a799-b47f040541d1" containerName="console"
Jul 17 09:34:42.532283 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.532239    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:34:42.532461 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.532353    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.535224 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.535193    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 09:34:42.554184 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.554148    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:34:42.681125 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.681073    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmqmr\" (UniqueName: \"kubernetes.io/projected/2be2706b-989b-40d8-babb-237b01f5c399-kube-api-access-fmqmr\") pod \"limitador-limitador-54d6d988d5-bcxhk\" (UID: \"2be2706b-989b-40d8-babb-237b01f5c399\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.681303 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.681130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2706b-989b-40d8-babb-237b01f5c399-config-file\") pod \"limitador-limitador-54d6d988d5-bcxhk\" (UID: \"2be2706b-989b-40d8-babb-237b01f5c399\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.782525 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.782419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmqmr\" (UniqueName: \"kubernetes.io/projected/2be2706b-989b-40d8-babb-237b01f5c399-kube-api-access-fmqmr\") pod \"limitador-limitador-54d6d988d5-bcxhk\" (UID: \"2be2706b-989b-40d8-babb-237b01f5c399\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.782525 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.782482    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2706b-989b-40d8-babb-237b01f5c399-config-file\") pod \"limitador-limitador-54d6d988d5-bcxhk\" (UID: \"2be2706b-989b-40d8-babb-237b01f5c399\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.783253 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.783228    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2be2706b-989b-40d8-babb-237b01f5c399-config-file\") pod \"limitador-limitador-54d6d988d5-bcxhk\" (UID: \"2be2706b-989b-40d8-babb-237b01f5c399\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.791433 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.791396    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmqmr\" (UniqueName: \"kubernetes.io/projected/2be2706b-989b-40d8-babb-237b01f5c399-kube-api-access-fmqmr\") pod \"limitador-limitador-54d6d988d5-bcxhk\" (UID: \"2be2706b-989b-40d8-babb-237b01f5c399\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.843585 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.843538    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:42.918808 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.918775    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-pxksl"]
Jul 17 09:34:42.923098 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.923071    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:42.925624 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.925594    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-vn8cp\""
Jul 17 09:34:42.930126 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.930092    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pxksl"]
Jul 17 09:34:42.954188 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.954153    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:42.994840 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:42.994792    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:34:42.998570 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:34:42.998532    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be2706b_989b_40d8_babb_237b01f5c399.slice/crio-3c4d8bd7023dc5f09e0a296822f0b8fdaedea9a90b0d3928cc1e410e8832e3a4 WatchSource:0}: Error finding container 3c4d8bd7023dc5f09e0a296822f0b8fdaedea9a90b0d3928cc1e410e8832e3a4: Status 404 returned error can't find the container with id 3c4d8bd7023dc5f09e0a296822f0b8fdaedea9a90b0d3928cc1e410e8832e3a4
Jul 17 09:34:43.015174 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.015128    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:43.042717 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.042632    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dvkp"]
Jul 17 09:34:43.046905 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.046881    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:34:43.053363 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.053332    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dvkp"]
Jul 17 09:34:43.085039 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.085002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4wdh\" (UniqueName: \"kubernetes.io/projected/c1f97a8b-076f-430b-9ed9-e803667cf650-kube-api-access-p4wdh\") pod \"authorino-58cd955675-pxksl\" (UID: \"c1f97a8b-076f-430b-9ed9-e803667cf650\") " pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:43.185936 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.185890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4wdh\" (UniqueName: \"kubernetes.io/projected/c1f97a8b-076f-430b-9ed9-e803667cf650-kube-api-access-p4wdh\") pod \"authorino-58cd955675-pxksl\" (UID: \"c1f97a8b-076f-430b-9ed9-e803667cf650\") " pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:43.186357 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.185973    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8jkb\" (UniqueName: \"kubernetes.io/projected/225d128d-06d3-4aa7-8eb4-b102a597c73e-kube-api-access-l8jkb\") pod \"authorino-75d556bd77-9dvkp\" (UID: \"225d128d-06d3-4aa7-8eb4-b102a597c73e\") " pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:34:43.197487 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.197441    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4wdh\" (UniqueName: \"kubernetes.io/projected/c1f97a8b-076f-430b-9ed9-e803667cf650-kube-api-access-p4wdh\") pod \"authorino-58cd955675-pxksl\" (UID: \"c1f97a8b-076f-430b-9ed9-e803667cf650\") " pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:43.235457 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.235415    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:43.287069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.287000    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8jkb\" (UniqueName: \"kubernetes.io/projected/225d128d-06d3-4aa7-8eb4-b102a597c73e-kube-api-access-l8jkb\") pod \"authorino-75d556bd77-9dvkp\" (UID: \"225d128d-06d3-4aa7-8eb4-b102a597c73e\") " pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:34:43.297138 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.296755    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8jkb\" (UniqueName: \"kubernetes.io/projected/225d128d-06d3-4aa7-8eb4-b102a597c73e-kube-api-access-l8jkb\") pod \"authorino-75d556bd77-9dvkp\" (UID: \"225d128d-06d3-4aa7-8eb4-b102a597c73e\") " pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:34:43.358219 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.358176    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:34:43.390687 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.390632    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-pxksl"]
Jul 17 09:34:43.394808 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:34:43.394768    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1f97a8b_076f_430b_9ed9_e803667cf650.slice/crio-1408c0919503daca23052562da7f9853b37a1cdf66e36d4dc357e21883b8718d WatchSource:0}: Error finding container 1408c0919503daca23052562da7f9853b37a1cdf66e36d4dc357e21883b8718d: Status 404 returned error can't find the container with id 1408c0919503daca23052562da7f9853b37a1cdf66e36d4dc357e21883b8718d
Jul 17 09:34:43.536685 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.536644    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dvkp"]
Jul 17 09:34:43.543608 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:34:43.543561    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod225d128d_06d3_4aa7_8eb4_b102a597c73e.slice/crio-7457e972752addecd164d5bd10608b08af561a283bfc350f64fca8402edb136d WatchSource:0}: Error finding container 7457e972752addecd164d5bd10608b08af561a283bfc350f64fca8402edb136d: Status 404 returned error can't find the container with id 7457e972752addecd164d5bd10608b08af561a283bfc350f64fca8402edb136d
Jul 17 09:34:43.562408 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.562365    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dvkp" event={"ID":"225d128d-06d3-4aa7-8eb4-b102a597c73e","Type":"ContainerStarted","Data":"7457e972752addecd164d5bd10608b08af561a283bfc350f64fca8402edb136d"}
Jul 17 09:34:43.564856 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.564759    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pxksl" event={"ID":"c1f97a8b-076f-430b-9ed9-e803667cf650","Type":"ContainerStarted","Data":"1408c0919503daca23052562da7f9853b37a1cdf66e36d4dc357e21883b8718d"}
Jul 17 09:34:43.566989 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:43.566877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk" event={"ID":"2be2706b-989b-40d8-babb-237b01f5c399","Type":"ContainerStarted","Data":"3c4d8bd7023dc5f09e0a296822f0b8fdaedea9a90b0d3928cc1e410e8832e3a4"}
Jul 17 09:34:46.571492 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:46.571462    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:46.671853 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:46.671804    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:46.966426 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:46.966378    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:47.071083 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:47.071043    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:48.598491 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.598450    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dvkp" event={"ID":"225d128d-06d3-4aa7-8eb4-b102a597c73e","Type":"ContainerStarted","Data":"2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c"}
Jul 17 09:34:48.599926 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.599901    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pxksl" event={"ID":"c1f97a8b-076f-430b-9ed9-e803667cf650","Type":"ContainerStarted","Data":"9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f"}
Jul 17 09:34:48.601330 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.601302    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk" event={"ID":"2be2706b-989b-40d8-babb-237b01f5c399","Type":"ContainerStarted","Data":"605a3b4af085f450c88ddc47503c30bf7d908c1cecb81beb95df6c5e81d82c31"}
Jul 17 09:34:48.601440 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.601367    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:34:48.621378 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.621324    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9dvkp" podStartSLOduration=1.170726263 podStartE2EDuration="5.62130886s" totalImagesPullingTime="4.450582597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:48.620967266 +0000 UTC m=+472.631091192" watchObservedRunningTime="2026-07-17 09:34:48.62130886 +0000 UTC m=+472.631432828"
Jul 17 09:34:48.642918 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.642868    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk" podStartSLOduration=1.662553889 podStartE2EDuration="6.642846728s" totalImagesPullingTime="4.980292839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:48.641653378 +0000 UTC m=+472.651777304" watchObservedRunningTime="2026-07-17 09:34:48.642846728 +0000 UTC m=+472.652970655"
Jul 17 09:34:48.649411 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.649377    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pxksl"]
Jul 17 09:34:48.661803 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:48.661728    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-pxksl" podStartSLOduration=2.123793714 podStartE2EDuration="6.661710986s" totalImagesPullingTime="4.537917272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:48.660954432 +0000 UTC m=+472.671078395" watchObservedRunningTime="2026-07-17 09:34:48.661710986 +0000 UTC m=+472.671834910"
Jul 17 09:34:49.385021 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:49.384952    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:49.485223 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:49.485185    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:50.610354 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:50.610297    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-pxksl" podUID="c1f97a8b-076f-430b-9ed9-e803667cf650" containerName="authorino" containerID="cri-o://9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f" gracePeriod=30
Jul 17 09:34:50.859717 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:50.859670    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:50.953336 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:50.953294    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4wdh\" (UniqueName: \"kubernetes.io/projected/c1f97a8b-076f-430b-9ed9-e803667cf650-kube-api-access-p4wdh\") pod \"c1f97a8b-076f-430b-9ed9-e803667cf650\" (UID: \"c1f97a8b-076f-430b-9ed9-e803667cf650\") "
Jul 17 09:34:50.955766 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:50.955736    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f97a8b-076f-430b-9ed9-e803667cf650-kube-api-access-p4wdh" (OuterVolumeSpecName: "kube-api-access-p4wdh") pod "c1f97a8b-076f-430b-9ed9-e803667cf650" (UID: "c1f97a8b-076f-430b-9ed9-e803667cf650"). InnerVolumeSpecName "kube-api-access-p4wdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:34:51.054512 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.054455    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4wdh\" (UniqueName: \"kubernetes.io/projected/c1f97a8b-076f-430b-9ed9-e803667cf650-kube-api-access-p4wdh\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:34:51.616633 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.616598    2556 generic.go:356] "Generic (PLEG): container finished" podID="c1f97a8b-076f-430b-9ed9-e803667cf650" containerID="9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f" exitCode=0
Jul 17 09:34:51.617073 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.616655    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-pxksl"
Jul 17 09:34:51.617073 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.616682    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pxksl" event={"ID":"c1f97a8b-076f-430b-9ed9-e803667cf650","Type":"ContainerDied","Data":"9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f"}
Jul 17 09:34:51.617073 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.616723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-pxksl" event={"ID":"c1f97a8b-076f-430b-9ed9-e803667cf650","Type":"ContainerDied","Data":"1408c0919503daca23052562da7f9853b37a1cdf66e36d4dc357e21883b8718d"}
Jul 17 09:34:51.617073 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.616743    2556 scope.go:117] "RemoveContainer" containerID="9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f"
Jul 17 09:34:51.628234 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.626873    2556 scope.go:117] "RemoveContainer" containerID="9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f"
Jul 17 09:34:51.628587 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:34:51.628551    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f\": container with ID starting with 9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f not found: ID does not exist" containerID="9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f"
Jul 17 09:34:51.628695 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.628601    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f"} err="failed to get container status \"9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f\": rpc error: code = NotFound desc = could not find container \"9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f\": container with ID starting with 9f19feb2578acda7e1964444ab95c89d7b78ff5b30f100a973e16e2f0ce9f66f not found: ID does not exist"
Jul 17 09:34:51.638329 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.638287    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-pxksl"]
Jul 17 09:34:51.643487 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:51.643458    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-pxksl"]
Jul 17 09:34:52.495638 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:52.495589    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f97a8b-076f-430b-9ed9-e803667cf650" path="/var/lib/kubelet/pods/c1f97a8b-076f-430b-9ed9-e803667cf650/volumes"
Jul 17 09:34:53.580318 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:53.580280    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:53.683885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:53.683813    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:34:59.608517 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:34:59.608484    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"
Jul 17 09:35:03.348370 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:03.348332    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:03.447397 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:03.447364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:03.838323 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:03.838280    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:15.561943 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.561908    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:15.780107 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.780073    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbjv"]
Jul 17 09:35:15.780372 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.780360    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f97a8b-076f-430b-9ed9-e803667cf650" containerName="authorino"
Jul 17 09:35:15.780471 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.780374    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f97a8b-076f-430b-9ed9-e803667cf650" containerName="authorino"
Jul 17 09:35:15.780471 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.780433    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f97a8b-076f-430b-9ed9-e803667cf650" containerName="authorino"
Jul 17 09:35:15.792274 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.792241    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:15.800386 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.800340    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbjv"]
Jul 17 09:35:15.800569 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.800456    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:15.843388 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.843291    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj9r2\" (UniqueName: \"kubernetes.io/projected/d6df79f7-002f-456e-b07d-ce20edc0a736-kube-api-access-rj9r2\") pod \"authorino-5d4b67c446-jjbjv\" (UID: \"d6df79f7-002f-456e-b07d-ce20edc0a736\") " pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:15.944159 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.944124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj9r2\" (UniqueName: \"kubernetes.io/projected/d6df79f7-002f-456e-b07d-ce20edc0a736-kube-api-access-rj9r2\") pod \"authorino-5d4b67c446-jjbjv\" (UID: \"d6df79f7-002f-456e-b07d-ce20edc0a736\") " pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:15.954814 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.954781    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj9r2\" (UniqueName: \"kubernetes.io/projected/d6df79f7-002f-456e-b07d-ce20edc0a736-kube-api-access-rj9r2\") pod \"authorino-5d4b67c446-jjbjv\" (UID: \"d6df79f7-002f-456e-b07d-ce20edc0a736\") " pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:15.966507 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.966468    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbjv"]
Jul 17 09:35:15.966717 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:15.966704    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:16.000458 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.000411    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-mt5vx"]
Jul 17 09:35:16.005032 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.004999    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.007202 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.007176    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 17 09:35:16.011607 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.011566    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-mt5vx"]
Jul 17 09:35:16.045495 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.045464    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-tls-cert\") pod \"authorino-76877994-mt5vx\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") " pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.045659 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.045515    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crnbd\" (UniqueName: \"kubernetes.io/projected/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-kube-api-access-crnbd\") pod \"authorino-76877994-mt5vx\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") " pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.054626 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.054592    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-mt5vx"]
Jul 17 09:35:16.054859 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:35:16.054838    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-crnbd tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-mt5vx" podUID="ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613"
Jul 17 09:35:16.087080 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.087037    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d4f4f884f-7lhfc"]
Jul 17 09:35:16.090425 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.090398    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.097524 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.097379    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d4f4f884f-7lhfc"]
Jul 17 09:35:16.112213 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.112188    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbjv"]
Jul 17 09:35:16.114071 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:35:16.114044    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6df79f7_002f_456e_b07d_ce20edc0a736.slice/crio-366bd1979516f336a18090c6adc4ed5fe1906dfcf9f87865a3fcc6ce92988404 WatchSource:0}: Error finding container 366bd1979516f336a18090c6adc4ed5fe1906dfcf9f87865a3fcc6ce92988404: Status 404 returned error can't find the container with id 366bd1979516f336a18090c6adc4ed5fe1906dfcf9f87865a3fcc6ce92988404
Jul 17 09:35:16.146219 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.146172    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c0531173-c1de-4cc6-b01d-ccdd881ed04e-tls-cert\") pod \"authorino-d4f4f884f-7lhfc\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") " pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.146399 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.146255    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-tls-cert\") pod \"authorino-76877994-mt5vx\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") " pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.146399 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.146288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzbqk\" (UniqueName: \"kubernetes.io/projected/c0531173-c1de-4cc6-b01d-ccdd881ed04e-kube-api-access-wzbqk\") pod \"authorino-d4f4f884f-7lhfc\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") " pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.146399 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.146316    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crnbd\" (UniqueName: \"kubernetes.io/projected/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-kube-api-access-crnbd\") pod \"authorino-76877994-mt5vx\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") " pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.149005 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.148971    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-tls-cert\") pod \"authorino-76877994-mt5vx\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") " pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.154572 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.154544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crnbd\" (UniqueName: \"kubernetes.io/projected/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-kube-api-access-crnbd\") pod \"authorino-76877994-mt5vx\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") " pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.247577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.247533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c0531173-c1de-4cc6-b01d-ccdd881ed04e-tls-cert\") pod \"authorino-d4f4f884f-7lhfc\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") " pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.247767 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.247604    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzbqk\" (UniqueName: \"kubernetes.io/projected/c0531173-c1de-4cc6-b01d-ccdd881ed04e-kube-api-access-wzbqk\") pod \"authorino-d4f4f884f-7lhfc\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") " pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.250239 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.250214    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c0531173-c1de-4cc6-b01d-ccdd881ed04e-tls-cert\") pod \"authorino-d4f4f884f-7lhfc\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") " pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.255225 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.255201    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzbqk\" (UniqueName: \"kubernetes.io/projected/c0531173-c1de-4cc6-b01d-ccdd881ed04e-kube-api-access-wzbqk\") pod \"authorino-d4f4f884f-7lhfc\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") " pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.400953 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.400909    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:35:16.542878 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.542697    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d4f4f884f-7lhfc"]
Jul 17 09:35:16.545414 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:35:16.545386    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0531173_c1de_4cc6_b01d_ccdd881ed04e.slice/crio-243a14e7a6dd0d8e2018bb24952c525dfd6ba8c48e8fd665cb3b1f98cda7c977 WatchSource:0}: Error finding container 243a14e7a6dd0d8e2018bb24952c525dfd6ba8c48e8fd665cb3b1f98cda7c977: Status 404 returned error can't find the container with id 243a14e7a6dd0d8e2018bb24952c525dfd6ba8c48e8fd665cb3b1f98cda7c977
Jul 17 09:35:16.725635 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.725535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbjv" event={"ID":"d6df79f7-002f-456e-b07d-ce20edc0a736","Type":"ContainerStarted","Data":"366bd1979516f336a18090c6adc4ed5fe1906dfcf9f87865a3fcc6ce92988404"}
Jul 17 09:35:16.726742 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.726722    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.726856 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.726723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4f4f884f-7lhfc" event={"ID":"c0531173-c1de-4cc6-b01d-ccdd881ed04e","Type":"ContainerStarted","Data":"243a14e7a6dd0d8e2018bb24952c525dfd6ba8c48e8fd665cb3b1f98cda7c977"}
Jul 17 09:35:16.735610 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.735563    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:16.852893 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.852848    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-tls-cert\") pod \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") "
Jul 17 09:35:16.853070 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.852903    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crnbd\" (UniqueName: \"kubernetes.io/projected/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-kube-api-access-crnbd\") pod \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\" (UID: \"ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613\") "
Jul 17 09:35:16.855262 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.855220    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613" (UID: "ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:35:16.855396 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.855259    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-kube-api-access-crnbd" (OuterVolumeSpecName: "kube-api-access-crnbd") pod "ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613" (UID: "ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613"). InnerVolumeSpecName "kube-api-access-crnbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:35:16.953719 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.953688    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-tls-cert\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:35:16.953856 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:16.953726    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crnbd\" (UniqueName: \"kubernetes.io/projected/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613-kube-api-access-crnbd\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:35:17.732351 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.732307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4f4f884f-7lhfc" event={"ID":"c0531173-c1de-4cc6-b01d-ccdd881ed04e","Type":"ContainerStarted","Data":"95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768"}
Jul 17 09:35:17.733788 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.733754    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbjv" event={"ID":"d6df79f7-002f-456e-b07d-ce20edc0a736","Type":"ContainerStarted","Data":"ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051"}
Jul 17 09:35:17.733788 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.733778    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-mt5vx"
Jul 17 09:35:17.734004 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.733801    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jjbjv" podUID="d6df79f7-002f-456e-b07d-ce20edc0a736" containerName="authorino" containerID="cri-o://ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051" gracePeriod=30
Jul 17 09:35:17.747932 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.747880    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d4f4f884f-7lhfc" podStartSLOduration=1.392287041 podStartE2EDuration="1.747861557s" totalImagesPullingTime="355.574516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:35:17.746704164 +0000 UTC m=+501.756828090" watchObservedRunningTime="2026-07-17 09:35:17.747861557 +0000 UTC m=+501.757985483"
Jul 17 09:35:17.768721 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.768688    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-mt5vx"]
Jul 17 09:35:17.773001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.772957    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dvkp"]
Jul 17 09:35:17.773162 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.773143    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9dvkp" podUID="225d128d-06d3-4aa7-8eb4-b102a597c73e" containerName="authorino" containerID="cri-o://2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c" gracePeriod=30
Jul 17 09:35:17.775630 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.775595    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-mt5vx"]
Jul 17 09:35:17.790294 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:17.790239    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jjbjv" podStartSLOduration=2.032554612 podStartE2EDuration="2.790221736s" totalImagesPullingTime="757.667124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:35:17.789428113 +0000 UTC m=+501.799552040" watchObservedRunningTime="2026-07-17 09:35:17.790221736 +0000 UTC m=+501.800345667"
Jul 17 09:35:18.062491 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.062459    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:18.065026 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.064985    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:35:18.163354 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.163311    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8jkb\" (UniqueName: \"kubernetes.io/projected/225d128d-06d3-4aa7-8eb4-b102a597c73e-kube-api-access-l8jkb\") pod \"225d128d-06d3-4aa7-8eb4-b102a597c73e\" (UID: \"225d128d-06d3-4aa7-8eb4-b102a597c73e\") "
Jul 17 09:35:18.163354 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.163357    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj9r2\" (UniqueName: \"kubernetes.io/projected/d6df79f7-002f-456e-b07d-ce20edc0a736-kube-api-access-rj9r2\") pod \"d6df79f7-002f-456e-b07d-ce20edc0a736\" (UID: \"d6df79f7-002f-456e-b07d-ce20edc0a736\") "
Jul 17 09:35:18.165772 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.165727    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6df79f7-002f-456e-b07d-ce20edc0a736-kube-api-access-rj9r2" (OuterVolumeSpecName: "kube-api-access-rj9r2") pod "d6df79f7-002f-456e-b07d-ce20edc0a736" (UID: "d6df79f7-002f-456e-b07d-ce20edc0a736"). InnerVolumeSpecName "kube-api-access-rj9r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:35:18.165926 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.165855    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225d128d-06d3-4aa7-8eb4-b102a597c73e-kube-api-access-l8jkb" (OuterVolumeSpecName: "kube-api-access-l8jkb") pod "225d128d-06d3-4aa7-8eb4-b102a597c73e" (UID: "225d128d-06d3-4aa7-8eb4-b102a597c73e"). InnerVolumeSpecName "kube-api-access-l8jkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:35:18.264266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.264216    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8jkb\" (UniqueName: \"kubernetes.io/projected/225d128d-06d3-4aa7-8eb4-b102a597c73e-kube-api-access-l8jkb\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:35:18.264266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.264258    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj9r2\" (UniqueName: \"kubernetes.io/projected/d6df79f7-002f-456e-b07d-ce20edc0a736-kube-api-access-rj9r2\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:35:18.496554 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.496472    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613" path="/var/lib/kubelet/pods/ac9bb1e9-37c0-4f7c-a5bb-81cad6e11613/volumes"
Jul 17 09:35:18.739959 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.739915    2556 generic.go:356] "Generic (PLEG): container finished" podID="d6df79f7-002f-456e-b07d-ce20edc0a736" containerID="ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051" exitCode=0
Jul 17 09:35:18.740391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.739973    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbjv"
Jul 17 09:35:18.740391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.740007    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbjv" event={"ID":"d6df79f7-002f-456e-b07d-ce20edc0a736","Type":"ContainerDied","Data":"ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051"}
Jul 17 09:35:18.740391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.740051    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbjv" event={"ID":"d6df79f7-002f-456e-b07d-ce20edc0a736","Type":"ContainerDied","Data":"366bd1979516f336a18090c6adc4ed5fe1906dfcf9f87865a3fcc6ce92988404"}
Jul 17 09:35:18.740391 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.740074    2556 scope.go:117] "RemoveContainer" containerID="ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051"
Jul 17 09:35:18.741545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.741325    2556 generic.go:356] "Generic (PLEG): container finished" podID="225d128d-06d3-4aa7-8eb4-b102a597c73e" containerID="2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c" exitCode=0
Jul 17 09:35:18.741545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.741384    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9dvkp"
Jul 17 09:35:18.741545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.741394    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dvkp" event={"ID":"225d128d-06d3-4aa7-8eb4-b102a597c73e","Type":"ContainerDied","Data":"2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c"}
Jul 17 09:35:18.741545 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.741441    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9dvkp" event={"ID":"225d128d-06d3-4aa7-8eb4-b102a597c73e","Type":"ContainerDied","Data":"7457e972752addecd164d5bd10608b08af561a283bfc350f64fca8402edb136d"}
Jul 17 09:35:18.748806 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.748782    2556 scope.go:117] "RemoveContainer" containerID="ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051"
Jul 17 09:35:18.749155 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:35:18.749125    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051\": container with ID starting with ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051 not found: ID does not exist" containerID="ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051"
Jul 17 09:35:18.749266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.749156    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051"} err="failed to get container status \"ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051\": rpc error: code = NotFound desc = could not find container \"ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051\": container with ID starting with ae52de1e2873236a9c4713ce6d834c049994d6e0a1414ecafc47ae673d92e051 not found: ID does not exist"
Jul 17 09:35:18.749266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.749174    2556 scope.go:117] "RemoveContainer" containerID="2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c"
Jul 17 09:35:18.758011 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.757972    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbjv"]
Jul 17 09:35:18.758636 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.758611    2556 scope.go:117] "RemoveContainer" containerID="2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c"
Jul 17 09:35:18.759013 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:35:18.758988    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c\": container with ID starting with 2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c not found: ID does not exist" containerID="2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c"
Jul 17 09:35:18.759120 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.759027    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c"} err="failed to get container status \"2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c\": rpc error: code = NotFound desc = could not find container \"2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c\": container with ID starting with 2f792a03cad18b97c4c507db4cfaff32cd645461d8358c942a0331deb982522c not found: ID does not exist"
Jul 17 09:35:18.759494 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.759475    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbjv"]
Jul 17 09:35:18.768745 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.768706    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dvkp"]
Jul 17 09:35:18.772531 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:18.772499    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9dvkp"]
Jul 17 09:35:20.495326 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:20.495290    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225d128d-06d3-4aa7-8eb4-b102a597c73e" path="/var/lib/kubelet/pods/225d128d-06d3-4aa7-8eb4-b102a597c73e/volumes"
Jul 17 09:35:20.495703 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:20.495603    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6df79f7-002f-456e-b07d-ce20edc0a736" path="/var/lib/kubelet/pods/d6df79f7-002f-456e-b07d-ce20edc0a736/volumes"
Jul 17 09:35:22.478800 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:22.478763    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:22.618270 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:22.618237    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:22.702041 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:22.702004    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:25.550982 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:25.550931    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:25.650253 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:25.650217    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:56.543265 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:56.543228    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:56.643262 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:56.643215    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:57.038964 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:57.038926    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:57.284689 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:57.284646    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:57.515385 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:57.515349    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:57.613144 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:57.613108    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:58.128063 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:58.128024    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:35:59.282398 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:59.282359    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:35:59.348837 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:35:59.348785    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:00.809948 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:00.809915    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:07.724996 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:07.724960    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:07.826755 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:07.826726    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:17.298357 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:17.298316    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:17.869069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:17.869032    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:36:33.495524 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:33.495482    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:33.967488 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:33.967452    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:36:50.995258 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:50.995209    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:36:52.066184 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:36:52.066144    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:37:15.301471 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:15.301422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:37:16.850711 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:16.850625    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:37:22.352381 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:22.352344    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:37:24.155177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:24.155132    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:37:34.221118 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:34.221076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:37:35.157788 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.157747    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"]
Jul 17 09:37:35.158100 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.158086    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6df79f7-002f-456e-b07d-ce20edc0a736" containerName="authorino"
Jul 17 09:37:35.158153 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.158102    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6df79f7-002f-456e-b07d-ce20edc0a736" containerName="authorino"
Jul 17 09:37:35.158153 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.158120    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225d128d-06d3-4aa7-8eb4-b102a597c73e" containerName="authorino"
Jul 17 09:37:35.158153 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.158125    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="225d128d-06d3-4aa7-8eb4-b102a597c73e" containerName="authorino"
Jul 17 09:37:35.158241 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.158177    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="225d128d-06d3-4aa7-8eb4-b102a597c73e" containerName="authorino"
Jul 17 09:37:35.158241 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.158187    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6df79f7-002f-456e-b07d-ce20edc0a736" containerName="authorino"
Jul 17 09:37:35.161137 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.161111    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.163426 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.163398    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 09:37:35.164241 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.164214    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-thd5v\""
Jul 17 09:37:35.164360 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.164259    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 09:37:35.164360 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.164292    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 17 09:37:35.172359 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.172328    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"]
Jul 17 09:37:35.184799 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.184763    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:37:35.243460 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.243415    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.243460 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.243465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.243936 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.243519    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33ef5c7a-9f1c-4175-8d10-a8f95053d113-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.243936 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.243543    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtn4c\" (UniqueName: \"kubernetes.io/projected/33ef5c7a-9f1c-4175-8d10-a8f95053d113-kube-api-access-jtn4c\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.243936 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.243591    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.243936 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.243621    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344025 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.343968    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344236 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344044    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33ef5c7a-9f1c-4175-8d10-a8f95053d113-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344236 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344082    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtn4c\" (UniqueName: \"kubernetes.io/projected/33ef5c7a-9f1c-4175-8d10-a8f95053d113-kube-api-access-jtn4c\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344236 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344122    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344406 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344299    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344406 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344353    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344533 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344461    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344663 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344632    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.344788 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.344703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.346845 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.346785    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33ef5c7a-9f1c-4175-8d10-a8f95053d113-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.347001 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.346984    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33ef5c7a-9f1c-4175-8d10-a8f95053d113-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.352206 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.352172    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtn4c\" (UniqueName: \"kubernetes.io/projected/33ef5c7a-9f1c-4175-8d10-a8f95053d113-kube-api-access-jtn4c\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr\" (UID: \"33ef5c7a-9f1c-4175-8d10-a8f95053d113\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.470768 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.470672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:35.612219 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.612189    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"]
Jul 17 09:37:35.614733 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:37:35.614695    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33ef5c7a_9f1c_4175_8d10_a8f95053d113.slice/crio-fcc53e915a93fbf2f5bdd30acb5a2ed6df687c3517fbe5c5993226521a4b975c WatchSource:0}: Error finding container fcc53e915a93fbf2f5bdd30acb5a2ed6df687c3517fbe5c5993226521a4b975c: Status 404 returned error can't find the container with id fcc53e915a93fbf2f5bdd30acb5a2ed6df687c3517fbe5c5993226521a4b975c
Jul 17 09:37:35.616524 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:35.616503    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:37:36.324975 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:36.324935    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr" event={"ID":"33ef5c7a-9f1c-4175-8d10-a8f95053d113","Type":"ContainerStarted","Data":"fcc53e915a93fbf2f5bdd30acb5a2ed6df687c3517fbe5c5993226521a4b975c"}
Jul 17 09:37:37.331451 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:37.331411    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr" event={"ID":"33ef5c7a-9f1c-4175-8d10-a8f95053d113","Type":"ContainerStarted","Data":"2810abe5af2d7c60e57bf3e68c4f9dc91efbac88d1ba4a3ec06286a56bfe4dc7"}
Jul 17 09:37:37.346629 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:37.346570    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr" podStartSLOduration=1.292305667 podStartE2EDuration="2.346554685s" totalImagesPullingTime="1.054249018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:37:37.345942668 +0000 UTC m=+641.356066592" watchObservedRunningTime="2026-07-17 09:37:37.346554685 +0000 UTC m=+641.356678608"
Jul 17 09:37:37.462731 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:37.462688    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:37:43.676160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:43.676121    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:37:45.471395 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:45.471353    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:45.471848 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:45.471414    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:45.484096 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:45.484062    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:37:46.383229 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:37:46.383194    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr"
Jul 17 09:38:19.151312 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:19.151277    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:38:19.252243 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:19.252206    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:38:24.105048 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.105009    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b98754ddd-wkw2s"]
Jul 17 09:38:24.108199 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.108177    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.115266 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.115233    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b98754ddd-wkw2s"]
Jul 17 09:38:24.256676 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.256630    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/459e981a-6020-4de5-8706-caf932594fa2-tls-cert\") pod \"authorino-6b98754ddd-wkw2s\" (UID: \"459e981a-6020-4de5-8706-caf932594fa2\") " pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.256676 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.256675    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm8bl\" (UniqueName: \"kubernetes.io/projected/459e981a-6020-4de5-8706-caf932594fa2-kube-api-access-hm8bl\") pod \"authorino-6b98754ddd-wkw2s\" (UID: \"459e981a-6020-4de5-8706-caf932594fa2\") " pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.357341 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.357229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/459e981a-6020-4de5-8706-caf932594fa2-tls-cert\") pod \"authorino-6b98754ddd-wkw2s\" (UID: \"459e981a-6020-4de5-8706-caf932594fa2\") " pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.357341 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.357290    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm8bl\" (UniqueName: \"kubernetes.io/projected/459e981a-6020-4de5-8706-caf932594fa2-kube-api-access-hm8bl\") pod \"authorino-6b98754ddd-wkw2s\" (UID: \"459e981a-6020-4de5-8706-caf932594fa2\") " pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.359975 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.359948    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/459e981a-6020-4de5-8706-caf932594fa2-tls-cert\") pod \"authorino-6b98754ddd-wkw2s\" (UID: \"459e981a-6020-4de5-8706-caf932594fa2\") " pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.364579 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.364548    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm8bl\" (UniqueName: \"kubernetes.io/projected/459e981a-6020-4de5-8706-caf932594fa2-kube-api-access-hm8bl\") pod \"authorino-6b98754ddd-wkw2s\" (UID: \"459e981a-6020-4de5-8706-caf932594fa2\") " pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.418243 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.418194    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b98754ddd-wkw2s"
Jul 17 09:38:24.542667 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.542629    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:38:24.594739 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.594708    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b98754ddd-wkw2s"]
Jul 17 09:38:24.597718 ip-10-0-141-244 kubenswrapper[2556]: W0717 09:38:24.597690    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod459e981a_6020_4de5_8706_caf932594fa2.slice/crio-a63c104d345d05df44dc9348dae88f1490264fffdaf674a06c7634f625dc1782 WatchSource:0}: Error finding container a63c104d345d05df44dc9348dae88f1490264fffdaf674a06c7634f625dc1782: Status 404 returned error can't find the container with id a63c104d345d05df44dc9348dae88f1490264fffdaf674a06c7634f625dc1782
Jul 17 09:38:24.659349 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:24.659306    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:38:25.544271 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.544233    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b98754ddd-wkw2s" event={"ID":"459e981a-6020-4de5-8706-caf932594fa2","Type":"ContainerStarted","Data":"fff0be26038716e74a93d664becd47308439df1f9611af2d822f84079826c0d9"}
Jul 17 09:38:25.544880 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.544284    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b98754ddd-wkw2s" event={"ID":"459e981a-6020-4de5-8706-caf932594fa2","Type":"ContainerStarted","Data":"a63c104d345d05df44dc9348dae88f1490264fffdaf674a06c7634f625dc1782"}
Jul 17 09:38:25.559902 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.559810    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b98754ddd-wkw2s" podStartSLOduration=1.0996602 podStartE2EDuration="1.559786444s" totalImagesPullingTime="460.126244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:38:25.558643936 +0000 UTC m=+689.568767866" watchObservedRunningTime="2026-07-17 09:38:25.559786444 +0000 UTC m=+689.569910369"
Jul 17 09:38:25.587473 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.587432    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d4f4f884f-7lhfc"]
Jul 17 09:38:25.587694 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.587667    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-d4f4f884f-7lhfc" podUID="c0531173-c1de-4cc6-b01d-ccdd881ed04e" containerName="authorino" containerID="cri-o://95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768" gracePeriod=30
Jul 17 09:38:25.841789 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.841747    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:38:25.863891 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.863855    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:38:25.928160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.928127    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:38:25.971267 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.971223    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c0531173-c1de-4cc6-b01d-ccdd881ed04e-tls-cert\") pod \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") "
Jul 17 09:38:25.971267 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.971272    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzbqk\" (UniqueName: \"kubernetes.io/projected/c0531173-c1de-4cc6-b01d-ccdd881ed04e-kube-api-access-wzbqk\") pod \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\" (UID: \"c0531173-c1de-4cc6-b01d-ccdd881ed04e\") "
Jul 17 09:38:25.973601 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.973558    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0531173-c1de-4cc6-b01d-ccdd881ed04e-kube-api-access-wzbqk" (OuterVolumeSpecName: "kube-api-access-wzbqk") pod "c0531173-c1de-4cc6-b01d-ccdd881ed04e" (UID: "c0531173-c1de-4cc6-b01d-ccdd881ed04e"). InnerVolumeSpecName "kube-api-access-wzbqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:38:25.982232 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:25.982194    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0531173-c1de-4cc6-b01d-ccdd881ed04e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c0531173-c1de-4cc6-b01d-ccdd881ed04e" (UID: "c0531173-c1de-4cc6-b01d-ccdd881ed04e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:38:26.072076 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.072028    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c0531173-c1de-4cc6-b01d-ccdd881ed04e-tls-cert\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:38:26.072076 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.072067    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzbqk\" (UniqueName: \"kubernetes.io/projected/c0531173-c1de-4cc6-b01d-ccdd881ed04e-kube-api-access-wzbqk\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 09:38:26.550284 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.550245    2556 generic.go:356] "Generic (PLEG): container finished" podID="c0531173-c1de-4cc6-b01d-ccdd881ed04e" containerID="95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768" exitCode=0
Jul 17 09:38:26.550733 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.550296    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d4f4f884f-7lhfc"
Jul 17 09:38:26.550733 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.550341    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4f4f884f-7lhfc" event={"ID":"c0531173-c1de-4cc6-b01d-ccdd881ed04e","Type":"ContainerDied","Data":"95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768"}
Jul 17 09:38:26.550733 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.550398    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d4f4f884f-7lhfc" event={"ID":"c0531173-c1de-4cc6-b01d-ccdd881ed04e","Type":"ContainerDied","Data":"243a14e7a6dd0d8e2018bb24952c525dfd6ba8c48e8fd665cb3b1f98cda7c977"}
Jul 17 09:38:26.550733 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.550420    2556 scope.go:117] "RemoveContainer" containerID="95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768"
Jul 17 09:38:26.560309 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.560285    2556 scope.go:117] "RemoveContainer" containerID="95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768"
Jul 17 09:38:26.560731 ip-10-0-141-244 kubenswrapper[2556]: E0717 09:38:26.560700    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768\": container with ID starting with 95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768 not found: ID does not exist" containerID="95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768"
Jul 17 09:38:26.560880 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.560745    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768"} err="failed to get container status \"95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768\": rpc error: code = NotFound desc = could not find container \"95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768\": container with ID starting with 95799185979fc353e10006125623e0c436ae59de0dda40d84f39b697d53ed768 not found: ID does not exist"
Jul 17 09:38:26.580105 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.580069    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d4f4f884f-7lhfc"]
Jul 17 09:38:26.586616 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:26.586588    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d4f4f884f-7lhfc"]
Jul 17 09:38:28.495540 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:28.495497    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0531173-c1de-4cc6-b01d-ccdd881ed04e" path="/var/lib/kubelet/pods/c0531173-c1de-4cc6-b01d-ccdd881ed04e/volumes"
Jul 17 09:38:52.951316 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:52.951234    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:38:56.051739 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:38:56.051698    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:39:04.751279 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:39:04.751237    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:39:13.869610 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:39:13.869565    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:39:18.501677 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:39:18.501624    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:39:18.563755 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:39:18.563707    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:39:46.770075 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:39:46.770028    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:40:29.357855 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:40:29.357795    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:40:30.575039 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:40:30.575003    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:40:56.072157 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:40:56.072113    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:41:06.208077 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:06.208041    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:06.308069 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:06.308024    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:06.469809 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:06.469694    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:06.866904 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:06.866046    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:41:12.019022 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:12.018983    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:12.129506 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:12.129465    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:12.219239 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:12.219204    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:12.391092 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:12.391052    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:18.042538 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:18.042494    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:18.143355 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:18.143322    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:41:34.766303 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:34.766258    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:41:44.556084 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:44.556043    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:41:55.274217 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:41:55.274128    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:42:04.353198 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:42:04.353157    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:42:15.063459 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:42:15.063420    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:42:23.560697 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:42:23.560648    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:42:35.660684 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:42:35.660635    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:42:46.561318 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:42:46.561275    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:43:07.780332 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:43:07.780291    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:43:12.067667 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:43:12.067622    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:43:23.649191 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:43:23.649147    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:43:45.061946 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:43:45.061910    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:43:56.668521 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:43:56.668486    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:44:12.865306 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:44:12.865266    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:44:22.069637 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:44:22.069593    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:44:32.667154 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:44:32.667108    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:44:41.671844 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:44:41.671416    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:44:52.261785 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:44:52.261687    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:45:44.599629 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:45:44.599588    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:46:38.569811 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:46:38.569762    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:46:50.560809 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:46:50.560764    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:46:59.260293 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:46:59.260256    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:47:11.268465 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:47:11.268411    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:47:21.462783 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:47:21.462742    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:47:26.790791 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:47:26.790751    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:47:32.665676 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:47:32.665637    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:47:41.966784 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:47:41.966726    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:47:52.757797 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:47:52.757707    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:48:03.065160 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:48:03.065124    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:48:13.362882 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:48:13.362813    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:48:25.264223 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:48:25.264184    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:48:47.663361 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:48:47.663323    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:48:56.667589 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:48:56.667548    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:49:06.861293 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:49:06.861250    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:49:17.957792 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:49:17.957701    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:49:35.363473 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:49:35.363432    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:49:43.280989 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:49:43.280948    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:49:44.851014 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:49:44.850970    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:49:54.458739 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:49:54.458696    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:50:02.753615 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:02.753575    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:50:12.367191 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:12.367145    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:50:21.974978 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:21.974937    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:50:35.606789 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:35.606744    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:50:37.761177 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:37.761141    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:50:45.065152 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:45.065111    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:50:54.761686 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:50:54.761577    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:51:05.264947 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:51:05.264906    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:51:16.665250 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:51:16.665205    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:51:23.456558 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:51:23.456518    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:51:42.064325 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:51:42.064279    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:51:51.171152 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:51:51.171115    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:52:01.772316 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:01.772278    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:52:10.568582 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:10.568542    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:52:15.452903 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:15.452864    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:15.541342 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:15.541307    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:20.903198 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:20.903164    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:20.964734 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:20.964702    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:21.121289 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:21.121226    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:21.194142 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:21.194051    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:22.308424 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:22.308388    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:22.394964 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:22.394905    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:22.471577 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:22.471544    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:22.701501 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:22.701469    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:22.906807 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:22.906748    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:23.007275 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:23.007183    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:24.827162 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:24.827132    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:52:25.055327 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:52:25.055292    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:54:15.269436 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:54:15.269336    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:54:15.326627 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:54:15.326589    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:54:27.525168 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:54:27.525125    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:28.395620 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:28.395579    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:29.866052 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:29.866012    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:31.053154 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:31.053117    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:55:33.791580 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:33.791541    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:34.372720 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:34.372679    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:40.000537 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:40.000495    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:40.061953 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:40.061917    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:40.271145 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:40.271048    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:40.368759 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:40.368727    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:40.519910 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:40.519879    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:41.016890 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:41.016859    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:41.116757 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:41.116721    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:41.728779 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:41.728733    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:41.874573 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:41.874534    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:42.204981 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:42.204947    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:42.328682 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:42.328650    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:42.430963 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:42.430932    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:44.268162 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:44.268123    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:55:44.378008 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:44.377974    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:52.980104 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:52.980061    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:55:53.076535 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:55:53.076502    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:04.184294 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:04.184250    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:04.290618 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:04.290578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:47.380516 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:47.380421    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:47.480538 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:47.480492    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:50.949534 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:50.949497    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:56:52.940496 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:52.940342    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:53.435306 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:53.435267    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:54.214852 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:54.214800    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:58.979230 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:58.979173    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:59.078597 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:59.078550    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:59.224880 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:59.224842    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:56:59.326580 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:56:59.326480    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:00.428083 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:00.428041    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:00.518471 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:00.518432    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:00.587895 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:00.587851    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:00.774523 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:00.774435    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:00.874388 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:00.874356    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:01.073885 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:01.073777    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:01.174989 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:01.174942    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:02.580116 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:02.580060    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:57:04.117721 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:57:04.117678    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 09:59:50.279719 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:59:50.279615    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 09:59:50.351580 ip-10-0-141-244 kubenswrapper[2556]: I0717 09:59:50.351540    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:00:34.462699 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:00:34.462657    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:00:37.016092 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:00:37.016060    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:00:55.784455 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:00:55.784391    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:00:55.989903 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:00:55.989865    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:00:56.747711 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:00:56.747666    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:00:56.879549 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:00:56.879498    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:07.766664 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:07.766591    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:07.826561 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:07.826529    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:10.670577 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:10.670537    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:03:13.877431 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:13.877396    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:13.975594 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:13.975558    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:19.386191 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:19.386153    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:19.487857 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:19.487805    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:19.658115 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:19.658020    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:20.809797 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:20.809743    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:20.919883 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:20.919853    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:21.001878 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:21.001835    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:21.253395 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:21.253364    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:21.583708 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:21.583623    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:21.685758 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:21.685724    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:23.363790 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:23.363755    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:03:29.484034 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:29.483999    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:29.584840 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:29.584782    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:03:33.784239 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:03:33.784211    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:05:24.066832 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:05:24.066756    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:05:55.460972 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:05:55.460890    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:05:56.791706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:05:56.791649    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:05.901319 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:05.901275    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:06.001481 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:06.001448    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:26.275243 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:26.275203    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:26.388139 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:26.388090    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:29.271464 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:29.271428    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:06:32.059533 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:32.059484    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:32.425424 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:32.425381    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:38.221844 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:38.221787    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:38.320308 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:38.320274    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:38.786063 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:38.786024    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:38.892085 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:38.892052    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:39.007326 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:39.007290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:40.535390 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:40.535350    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:42.961378 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:42.961345    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:46.771478 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:46.771436    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:46.897884 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:46.897834    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:47.020110 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:47.020069    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:47.512968 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:47.512932    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:54.272352 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:54.272307    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:06:54.368311 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:06:54.368275    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:03.896095 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:03.896053    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:13.592274 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:13.592236    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:19.256216 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:19.256127    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:19.362581 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:19.362538    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:19.462401 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:19.462347    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:35.385366 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:35.385335    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:42.190241 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:42.190194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:42.356113 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:42.356066    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:42.468803 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:42.468718    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:53.000470 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:53.000402    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:07:53.869053 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:07:53.869017    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:00.638145 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:00.638100    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:00.744554 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:00.744515    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:00.852722 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:00.852685    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:00.956239 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:00.956152    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:01.788868 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:01.788813    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:01.881663 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:01.881630    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:01.986940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:01.986899    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:03.635665 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:03.635629    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:04.466438 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:04.466401    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:08:09.670239 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:09.670201    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:09.770121 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:09.770065    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:10.385415 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:10.385380    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:14.063970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:14.063933    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:08:15.567493 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:15.567454    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:16.433365 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:16.433330    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:16.517775 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:16.517677    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:16.715069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:16.715031    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:17.106783 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:17.106740    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:17.218452 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:17.218411    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:20.230988 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:20.230956    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:20.381322 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:20.381292    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:20.507147 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:20.507059    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:20.693192 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:20.693147    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:21.315218 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:21.315182    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:21.416438 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:21.416400    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:24.598791 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:24.598746    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:25.323176 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:25.323138    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:25.421381 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:25.421345    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:25.771069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:25.771030    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:25.881005 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:25.880963    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:08:31.563184 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:31.563151    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:31.663372 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:31.663340    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:42.475700 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:42.475658    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:45.469553 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:45.469509    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:08:47.077140 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:47.077057    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:53.425335 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:53.425298    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:53.580475 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:53.580438    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:57.818859 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:57.818802    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:58.037841 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:58.037789    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:58.196055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:58.196018    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:58.275007 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:58.274973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:08:58.426506 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:08:58.426455    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:04.205537 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:04.205500    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:04.317344 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:04.317299    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:04.412086 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:04.412046    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:04.671057 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:04.671017    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:10.386474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:10.386422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:10.486658 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:10.486601    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:17.539636 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:17.539592    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:17.847963 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:17.847880    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:18.204837 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:18.204780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:21.240248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:21.240203    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:21.329440 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:21.329407    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:21.406986 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:21.406950    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:21.772942 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:21.772898    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:21.912035 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:21.911995    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:22.016172 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:22.016136    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:25.801938 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:25.801897    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:28.263062 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:28.263030    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:09:43.866192 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.866154    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"]
Jul 17 10:09:43.866758 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.866463    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0531173-c1de-4cc6-b01d-ccdd881ed04e" containerName="authorino"
Jul 17 10:09:43.866758 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.866475    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0531173-c1de-4cc6-b01d-ccdd881ed04e" containerName="authorino"
Jul 17 10:09:43.866758 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.866525    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0531173-c1de-4cc6-b01d-ccdd881ed04e" containerName="authorino"
Jul 17 10:09:43.869375 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.869354    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:43.872195 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.872172    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6bff328f\"/\"openshift-service-ca.crt\""
Jul 17 10:09:43.872605 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.872586    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6bff328f\"/\"kube-root-ca.crt\""
Jul 17 10:09:43.873146 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.873124    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6bff328f\"/\"e2e-lifecycle-model-6bff328f-kserve-self-signed-certs\""
Jul 17 10:09:43.873245 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.873194    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6bff328f\"/\"default-dockercfg-4g846\""
Jul 17 10:09:43.879387 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.879364    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"]
Jul 17 10:09:43.899964 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.899930    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:09:43.900179 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.900157    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-model-cache\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:43.900261 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.900200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tmp-dir\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:43.900324 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.900253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57qvg\" (UniqueName: \"kubernetes.io/projected/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-kube-api-access-57qvg\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:43.900324 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.900288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-dshm\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:43.900406 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.900324    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-home\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:43.900406 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:43.900374    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tls-certs\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.000750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.000713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tls-certs\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.000750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.000756    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-model-cache\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001012 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.000777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tmp-dir\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001012 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.000796    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57qvg\" (UniqueName: \"kubernetes.io/projected/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-kube-api-access-57qvg\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001012 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.000848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-dshm\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001012 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.000890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-home\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001237 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.001213    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-model-cache\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001308 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.001287    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tmp-dir\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.001350 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.001308    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-home\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.003323 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.003298    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-dshm\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.003499 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.003471    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tls-certs\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.011613 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.011585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57qvg\" (UniqueName: \"kubernetes.io/projected/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-kube-api-access-57qvg\") pod \"e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") " pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.178578 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.178543    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:44.311200 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.311174    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"]
Jul 17 10:09:44.313668 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:09:44.313642    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2cb626c_6a29_4a7f_ad08_74e8b0d00101.slice/crio-eb2dfacc93990b45b192ceab3684c4469f9e775389d21888d82a95662ce70948 WatchSource:0}: Error finding container eb2dfacc93990b45b192ceab3684c4469f9e775389d21888d82a95662ce70948: Status 404 returned error can't find the container with id eb2dfacc93990b45b192ceab3684c4469f9e775389d21888d82a95662ce70948
Jul 17 10:09:44.464409 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.464323    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d" event={"ID":"c2cb626c-6a29-4a7f-ad08-74e8b0d00101","Type":"ContainerStarted","Data":"60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642"}
Jul 17 10:09:44.464409 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.464366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d" event={"ID":"c2cb626c-6a29-4a7f-ad08-74e8b0d00101","Type":"ContainerStarted","Data":"eb2dfacc93990b45b192ceab3684c4469f9e775389d21888d82a95662ce70948"}
Jul 17 10:09:44.482006 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:44.481955    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d" podStartSLOduration=1.481940485 podStartE2EDuration="1.481940485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:44.480594011 +0000 UTC m=+2568.490717935" watchObservedRunningTime="2026-07-17 10:09:44.481940485 +0000 UTC m=+2568.492064409"
Jul 17 10:09:46.878218 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:46.878179    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:09:54.178963 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:54.178911    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:54.179443 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:54.179002    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:54.191120 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:54.191091    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:09:54.516519 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:09:54.516418    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:10:01.165893 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:01.165854    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:10:09.358458 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:09.358422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:10:09.458628 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:09.458593    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:10:14.655363 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:14.655320    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:10:18.368039 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:18.367953    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:10:19.252398 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:19.252368    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:10:24.222150 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:24.222105    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"]
Jul 17 10:10:24.319104 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:24.319063    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:10:39.243742 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.243691    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d" podUID="c2cb626c-6a29-4a7f-ad08-74e8b0d00101" containerName="main" containerID="cri-o://60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642" gracePeriod=45
Jul 17 10:10:39.502949 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.502876    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:10:39.579219 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579162    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tmp-dir\") pod \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") "
Jul 17 10:10:39.579514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579261    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tls-certs\") pod \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") "
Jul 17 10:10:39.579514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579283    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-dshm\") pod \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") "
Jul 17 10:10:39.579514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579301    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-model-cache\") pod \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") "
Jul 17 10:10:39.579514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579344    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57qvg\" (UniqueName: \"kubernetes.io/projected/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-kube-api-access-57qvg\") pod \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") "
Jul 17 10:10:39.579514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579364    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-home\") pod \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\" (UID: \"c2cb626c-6a29-4a7f-ad08-74e8b0d00101\") "
Jul 17 10:10:39.579868 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579602    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2cb626c-6a29-4a7f-ad08-74e8b0d00101" (UID: "c2cb626c-6a29-4a7f-ad08-74e8b0d00101"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:10:39.579868 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579682    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2cb626c-6a29-4a7f-ad08-74e8b0d00101" (UID: "c2cb626c-6a29-4a7f-ad08-74e8b0d00101"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:10:39.579868 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.579714    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-home" (OuterVolumeSpecName: "home") pod "c2cb626c-6a29-4a7f-ad08-74e8b0d00101" (UID: "c2cb626c-6a29-4a7f-ad08-74e8b0d00101"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:10:39.582500 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.582458    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2cb626c-6a29-4a7f-ad08-74e8b0d00101" (UID: "c2cb626c-6a29-4a7f-ad08-74e8b0d00101"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:10:39.582654 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.582541    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-dshm" (OuterVolumeSpecName: "dshm") pod "c2cb626c-6a29-4a7f-ad08-74e8b0d00101" (UID: "c2cb626c-6a29-4a7f-ad08-74e8b0d00101"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:10:39.582654 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.582562    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-kube-api-access-57qvg" (OuterVolumeSpecName: "kube-api-access-57qvg") pod "c2cb626c-6a29-4a7f-ad08-74e8b0d00101" (UID: "c2cb626c-6a29-4a7f-ad08-74e8b0d00101"). InnerVolumeSpecName "kube-api-access-57qvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:10:39.680745 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.680701    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:10:39.680745 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.680738    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:10:39.680975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.680766    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:10:39.680975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.680775    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57qvg\" (UniqueName: \"kubernetes.io/projected/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-kube-api-access-57qvg\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:10:39.680975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.680802    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:10:39.680975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.680812    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2cb626c-6a29-4a7f-ad08-74e8b0d00101-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:10:39.697469 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.697426    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2cb626c-6a29-4a7f-ad08-74e8b0d00101" containerID="60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642" exitCode=0
Jul 17 10:10:39.697622 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.697489    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"
Jul 17 10:10:39.697622 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.697500    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d" event={"ID":"c2cb626c-6a29-4a7f-ad08-74e8b0d00101","Type":"ContainerDied","Data":"60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642"}
Jul 17 10:10:39.697622 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.697539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d" event={"ID":"c2cb626c-6a29-4a7f-ad08-74e8b0d00101","Type":"ContainerDied","Data":"eb2dfacc93990b45b192ceab3684c4469f9e775389d21888d82a95662ce70948"}
Jul 17 10:10:39.697622 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.697561    2556 scope.go:117] "RemoveContainer" containerID="60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642"
Jul 17 10:10:39.706398 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.706334    2556 scope.go:117] "RemoveContainer" containerID="60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642"
Jul 17 10:10:39.706634 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:10:39.706612    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642\": container with ID starting with 60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642 not found: ID does not exist" containerID="60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642"
Jul 17 10:10:39.706676 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.706643    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642"} err="failed to get container status \"60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642\": rpc error: code = NotFound desc = could not find container \"60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642\": container with ID starting with 60766203bbb175441ec035a9038482b119a70dacb4730f624b9050f261489642 not found: ID does not exist"
Jul 17 10:10:39.729222 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.729194    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"]
Jul 17 10:10:39.730307 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:39.730288    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6bff328f/e2e-lifecycle-model-6bff328f-kserve-cc66f9895-nkg7d"]
Jul 17 10:10:40.494887 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:40.494849    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2cb626c-6a29-4a7f-ad08-74e8b0d00101" path="/var/lib/kubelet/pods/c2cb626c-6a29-4a7f-ad08-74e8b0d00101/volumes"
Jul 17 10:10:56.320762 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:56.320724    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:10:56.417669 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:10:56.417624    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:00.587697 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:00.587635    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:01.915433 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:01.915397    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:07.059202 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:07.059157    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:07.127887 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:07.127817    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:11.597119 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:11.597076    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:11.822123 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:11.822080    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:12.931039 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:12.930996    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:13.029280 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:13.029240    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:13.128692 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:13.128648    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:13.876931 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:13.876902    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:13.973267 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:13.973232    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:14.077896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:14.077855    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:15.474456 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:15.474419    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:17.687025 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:17.686983    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:11:21.832406 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:21.832353    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:21.917564 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:21.917528    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:22.536753 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:22.536714    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:28.610318 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:28.610264    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:28.714843 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:28.714781    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:28.815760 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:28.815719    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:29.658905 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:29.658871    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:11:39.347541 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:39.347504    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:11:39.711119 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:11:39.711073    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:00.689595 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:00.689506    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:06.333352 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:06.333314    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:06.400560 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:06.400511    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:06.680457 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:06.680391    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:06.993462 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:06.993352    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:07.199997 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:07.199958    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:10.056525 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:10.056494    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:10.152366 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:10.152330    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:10.297350 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:10.297315    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:10.766324 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:10.766286    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:11.051178 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:11.051084    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:11.153303 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:11.153252    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:14.549560 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:14.549515    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:14.902748 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:14.902709    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:15.227966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:15.227862    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:15.523206 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:15.523105    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:18.240750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:18.240707    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:18.337061 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:18.337022    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:18.594934 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:18.594841    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:18.689516 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:18.689479    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:12:18.704521 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:18.704478    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:19.367470 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:19.367403    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:19.473360 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:19.473316    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:22.746816 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:22.746780    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:31.837547 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:31.837505    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:31.857158 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:31.857121    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:12:31.933599 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:31.933561    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:39.947440 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.947403    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"]
Jul 17 10:12:39.948207 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.947703    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2cb626c-6a29-4a7f-ad08-74e8b0d00101" containerName="main"
Jul 17 10:12:39.948207 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.947715    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2cb626c-6a29-4a7f-ad08-74e8b0d00101" containerName="main"
Jul 17 10:12:39.948207 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.947766    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2cb626c-6a29-4a7f-ad08-74e8b0d00101" containerName="main"
Jul 17 10:12:39.950722 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.950701    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:39.953344 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.953314    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-21e67a\"/\"openshift-service-ca.crt\""
Jul 17 10:12:39.954485 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.954453    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-21e67a\"/\"default-dockercfg-dxshm\""
Jul 17 10:12:39.954631 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.954485    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-21e67a\"/\"kube-root-ca.crt\""
Jul 17 10:12:39.954631 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.954562    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-21e67a\"/\"test-model-21e67a-kserve-self-signed-certs\""
Jul 17 10:12:39.965521 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.965481    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"]
Jul 17 10:12:39.980514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:39.980449    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:40.097440 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.097402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-model-cache\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.097440 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.097447    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjwwt\" (UniqueName: \"kubernetes.io/projected/15cc23ea-caa0-4a20-8289-522550d5c27a-kube-api-access-bjwwt\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.097735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.097576    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-dshm\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.097735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.097617    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-home\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.097735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.097642    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15cc23ea-caa0-4a20-8289-522550d5c27a-tls-certs\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.097735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.097682    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-tmp-dir\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.198947 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.198857    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-dshm\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.198947 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.198897    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-home\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.198947 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.198914    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15cc23ea-caa0-4a20-8289-522550d5c27a-tls-certs\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.198947 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.198937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-tmp-dir\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.199291 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.198963    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-model-cache\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.199291 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.198988    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjwwt\" (UniqueName: \"kubernetes.io/projected/15cc23ea-caa0-4a20-8289-522550d5c27a-kube-api-access-bjwwt\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.199411 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.199382    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-tmp-dir\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.199473 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.199397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-home\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.199473 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.199457    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-model-cache\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.201515 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.201478    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-dshm\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.201757 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.201738    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15cc23ea-caa0-4a20-8289-522550d5c27a-tls-certs\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.208234 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.208200    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjwwt\" (UniqueName: \"kubernetes.io/projected/15cc23ea-caa0-4a20-8289-522550d5c27a-kube-api-access-bjwwt\") pod \"test-model-21e67a-kserve-764d8ffb5c-g4wnh\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") " pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.261265 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.261220    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:40.409505 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:40.409427    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"]
Jul 17 10:12:40.412121 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:12:40.412080    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15cc23ea_caa0_4a20_8289_522550d5c27a.slice/crio-fa9a17849e8a572d0b10915090448a7b6a2f677d20501d878b7be99e4e9e4a43 WatchSource:0}: Error finding container fa9a17849e8a572d0b10915090448a7b6a2f677d20501d878b7be99e4e9e4a43: Status 404 returned error can't find the container with id fa9a17849e8a572d0b10915090448a7b6a2f677d20501d878b7be99e4e9e4a43
Jul 17 10:12:41.214213 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:41.214174    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh" event={"ID":"15cc23ea-caa0-4a20-8289-522550d5c27a","Type":"ContainerStarted","Data":"12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e"}
Jul 17 10:12:41.214213 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:41.214219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh" event={"ID":"15cc23ea-caa0-4a20-8289-522550d5c27a","Type":"ContainerStarted","Data":"fa9a17849e8a572d0b10915090448a7b6a2f677d20501d878b7be99e4e9e4a43"}
Jul 17 10:12:41.236989 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:41.236935    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh" podStartSLOduration=2.236918316 podStartE2EDuration="2.236918316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:41.235662169 +0000 UTC m=+2745.245786106" watchObservedRunningTime="2026-07-17 10:12:41.236918316 +0000 UTC m=+2745.247042232"
Jul 17 10:12:41.380305 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:41.380267    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:41.479693 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:41.479602    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:12:44.965300 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:44.965258    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:12:50.261911 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:50.261867    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:50.262417 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:50.261930    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:50.274726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:50.274695    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:51.276469 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:51.276424    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:12:57.155887 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:12:57.155839    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:13:33.247318 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.247280    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"]
Jul 17 10:13:33.250486 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.250449    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.254143 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.254117    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1c996b\"/\"kube-root-ca.crt\""
Jul 17 10:13:33.254285 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.254144    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1c996b\"/\"test-model-1c996b-kserve-self-signed-certs\""
Jul 17 10:13:33.254285 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.254117    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1c996b\"/\"openshift-service-ca.crt\""
Jul 17 10:13:33.254285 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.254201    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1c996b\"/\"default-dockercfg-b78wf\""
Jul 17 10:13:33.262566 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.262538    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"]
Jul 17 10:13:33.288060 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.288023    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:13:33.337078 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.337026    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-home\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.337264 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.337091    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-tmp-dir\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.337264 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.337147    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-model-cache\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.337264 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.337168    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5sgb\" (UniqueName: \"kubernetes.io/projected/73296b49-caa5-4885-be3e-168507d8be3a-kube-api-access-d5sgb\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.337264 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.337248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73296b49-caa5-4885-be3e-168507d8be3a-tls-certs\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.337406 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.337302    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-dshm\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438120 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438085    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-home\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438291 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-tmp-dir\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438291 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-model-cache\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438379 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438308    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5sgb\" (UniqueName: \"kubernetes.io/projected/73296b49-caa5-4885-be3e-168507d8be3a-kube-api-access-d5sgb\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438379 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438348    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73296b49-caa5-4885-be3e-168507d8be3a-tls-certs\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438383    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-dshm\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438471    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-tmp-dir\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438603 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438582    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-home\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.438661 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.438616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-model-cache\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.440794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.440767    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-dshm\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.441035 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.441017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73296b49-caa5-4885-be3e-168507d8be3a-tls-certs\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.447564 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.447529    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5sgb\" (UniqueName: \"kubernetes.io/projected/73296b49-caa5-4885-be3e-168507d8be3a-kube-api-access-d5sgb\") pod \"test-model-1c996b-kserve-fc9dc94f-nmtkw\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") " pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.561720 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.561618    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:33.705082 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:33.705044    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"]
Jul 17 10:13:33.708257 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:13:33.708224    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73296b49_caa5_4885_be3e_168507d8be3a.slice/crio-da07bdb284ddcd202c4af37c1e47e51f3ecca0365436efee6691511fbd161c65 WatchSource:0}: Error finding container da07bdb284ddcd202c4af37c1e47e51f3ecca0365436efee6691511fbd161c65: Status 404 returned error can't find the container with id da07bdb284ddcd202c4af37c1e47e51f3ecca0365436efee6691511fbd161c65
Jul 17 10:13:34.441237 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:34.441192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw" event={"ID":"73296b49-caa5-4885-be3e-168507d8be3a","Type":"ContainerStarted","Data":"9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1"}
Jul 17 10:13:34.441237 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:34.441240    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw" event={"ID":"73296b49-caa5-4885-be3e-168507d8be3a","Type":"ContainerStarted","Data":"da07bdb284ddcd202c4af37c1e47e51f3ecca0365436efee6691511fbd161c65"}
Jul 17 10:13:34.458230 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:34.458171    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw" podStartSLOduration=1.458153623 podStartE2EDuration="1.458153623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:13:34.456354165 +0000 UTC m=+2798.466478090" watchObservedRunningTime="2026-07-17 10:13:34.458153623 +0000 UTC m=+2798.468277548"
Jul 17 10:13:37.572758 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:37.572721    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:13:43.561958 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:43.561913    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:43.561958 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:43.561972    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:43.574595 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:43.574558    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:44.499741 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:44.499710    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:13:52.466120 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:52.466077    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:13:52.779178 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:52.779081    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"]
Jul 17 10:13:52.801256 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:52.801218    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:13:54.005412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:54.005362    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"]
Jul 17 10:13:54.026589 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:54.026516    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:13:54.884536 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:54.884486    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:13:54.985679 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:13:54.985626    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:04.694557 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:04.694516    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:05.363422 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:05.363383    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:14:07.801297 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:07.801253    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh" podUID="15cc23ea-caa0-4a20-8289-522550d5c27a" containerName="main" containerID="cri-o://12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e" gracePeriod=45
Jul 17 10:14:08.054018 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.053953    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:14:08.149539 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149488    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-model-cache\") pod \"15cc23ea-caa0-4a20-8289-522550d5c27a\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") "
Jul 17 10:14:08.149539 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149542    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjwwt\" (UniqueName: \"kubernetes.io/projected/15cc23ea-caa0-4a20-8289-522550d5c27a-kube-api-access-bjwwt\") pod \"15cc23ea-caa0-4a20-8289-522550d5c27a\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") "
Jul 17 10:14:08.149772 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149576    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-dshm\") pod \"15cc23ea-caa0-4a20-8289-522550d5c27a\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") "
Jul 17 10:14:08.149772 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149601    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-home\") pod \"15cc23ea-caa0-4a20-8289-522550d5c27a\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") "
Jul 17 10:14:08.149901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149813    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15cc23ea-caa0-4a20-8289-522550d5c27a-tls-certs\") pod \"15cc23ea-caa0-4a20-8289-522550d5c27a\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") "
Jul 17 10:14:08.149901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149884    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-model-cache" (OuterVolumeSpecName: "model-cache") pod "15cc23ea-caa0-4a20-8289-522550d5c27a" (UID: "15cc23ea-caa0-4a20-8289-522550d5c27a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:08.150006 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149897    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-home" (OuterVolumeSpecName: "home") pod "15cc23ea-caa0-4a20-8289-522550d5c27a" (UID: "15cc23ea-caa0-4a20-8289-522550d5c27a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:08.150006 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.149895    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-tmp-dir\") pod \"15cc23ea-caa0-4a20-8289-522550d5c27a\" (UID: \"15cc23ea-caa0-4a20-8289-522550d5c27a\") "
Jul 17 10:14:08.150109 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.150086    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15cc23ea-caa0-4a20-8289-522550d5c27a" (UID: "15cc23ea-caa0-4a20-8289-522550d5c27a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:08.150228 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.150207    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:08.150228 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.150229    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:08.150408 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.150241    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:08.152108 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.152086    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-dshm" (OuterVolumeSpecName: "dshm") pod "15cc23ea-caa0-4a20-8289-522550d5c27a" (UID: "15cc23ea-caa0-4a20-8289-522550d5c27a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:08.152108 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.152091    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15cc23ea-caa0-4a20-8289-522550d5c27a-kube-api-access-bjwwt" (OuterVolumeSpecName: "kube-api-access-bjwwt") pod "15cc23ea-caa0-4a20-8289-522550d5c27a" (UID: "15cc23ea-caa0-4a20-8289-522550d5c27a"). InnerVolumeSpecName "kube-api-access-bjwwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:14:08.152280 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.152125    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15cc23ea-caa0-4a20-8289-522550d5c27a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15cc23ea-caa0-4a20-8289-522550d5c27a" (UID: "15cc23ea-caa0-4a20-8289-522550d5c27a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:14:08.251604 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.251511    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjwwt\" (UniqueName: \"kubernetes.io/projected/15cc23ea-caa0-4a20-8289-522550d5c27a-kube-api-access-bjwwt\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:08.251604 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.251599    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15cc23ea-caa0-4a20-8289-522550d5c27a-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:08.251604 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.251612    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15cc23ea-caa0-4a20-8289-522550d5c27a-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:08.590349 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.590313    2556 generic.go:356] "Generic (PLEG): container finished" podID="15cc23ea-caa0-4a20-8289-522550d5c27a" containerID="12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e" exitCode=0
Jul 17 10:14:08.590523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.590360    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"
Jul 17 10:14:08.590523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.590399    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh" event={"ID":"15cc23ea-caa0-4a20-8289-522550d5c27a","Type":"ContainerDied","Data":"12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e"}
Jul 17 10:14:08.590523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.590444    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh" event={"ID":"15cc23ea-caa0-4a20-8289-522550d5c27a","Type":"ContainerDied","Data":"fa9a17849e8a572d0b10915090448a7b6a2f677d20501d878b7be99e4e9e4a43"}
Jul 17 10:14:08.590523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.590462    2556 scope.go:117] "RemoveContainer" containerID="12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e"
Jul 17 10:14:08.599252 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.599229    2556 scope.go:117] "RemoveContainer" containerID="12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e"
Jul 17 10:14:08.599583 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:08.599564    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e\": container with ID starting with 12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e not found: ID does not exist" containerID="12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e"
Jul 17 10:14:08.599626 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.599595    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e"} err="failed to get container status \"12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e\": rpc error: code = NotFound desc = could not find container \"12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e\": container with ID starting with 12955f464ab2045a751300335c06fcdd1113ab3058149867b6831d9e5cef788e not found: ID does not exist"
Jul 17 10:14:08.606627 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.606588    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"]
Jul 17 10:14:08.610715 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:08.610684    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-21e67a/test-model-21e67a-kserve-764d8ffb5c-g4wnh"]
Jul 17 10:14:09.027616 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.027546    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw" podUID="73296b49-caa5-4885-be3e-168507d8be3a" containerName="main" containerID="cri-o://9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1" gracePeriod=45
Jul 17 10:14:09.280255 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.280184    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:14:09.307490 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.307457    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:09.360403 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360359    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5sgb\" (UniqueName: \"kubernetes.io/projected/73296b49-caa5-4885-be3e-168507d8be3a-kube-api-access-d5sgb\") pod \"73296b49-caa5-4885-be3e-168507d8be3a\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") "
Jul 17 10:14:09.360610 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360417    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73296b49-caa5-4885-be3e-168507d8be3a-tls-certs\") pod \"73296b49-caa5-4885-be3e-168507d8be3a\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") "
Jul 17 10:14:09.360610 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360457    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-home\") pod \"73296b49-caa5-4885-be3e-168507d8be3a\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") "
Jul 17 10:14:09.360610 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360479    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-dshm\") pod \"73296b49-caa5-4885-be3e-168507d8be3a\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") "
Jul 17 10:14:09.360610 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360519    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-model-cache\") pod \"73296b49-caa5-4885-be3e-168507d8be3a\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") "
Jul 17 10:14:09.360610 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360566    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-tmp-dir\") pod \"73296b49-caa5-4885-be3e-168507d8be3a\" (UID: \"73296b49-caa5-4885-be3e-168507d8be3a\") "
Jul 17 10:14:09.360963 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360804    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-model-cache" (OuterVolumeSpecName: "model-cache") pod "73296b49-caa5-4885-be3e-168507d8be3a" (UID: "73296b49-caa5-4885-be3e-168507d8be3a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:09.360963 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360815    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-home" (OuterVolumeSpecName: "home") pod "73296b49-caa5-4885-be3e-168507d8be3a" (UID: "73296b49-caa5-4885-be3e-168507d8be3a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:09.360963 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.360933    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73296b49-caa5-4885-be3e-168507d8be3a" (UID: "73296b49-caa5-4885-be3e-168507d8be3a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:09.363078 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.363042    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-dshm" (OuterVolumeSpecName: "dshm") pod "73296b49-caa5-4885-be3e-168507d8be3a" (UID: "73296b49-caa5-4885-be3e-168507d8be3a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:09.363190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.363088    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73296b49-caa5-4885-be3e-168507d8be3a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73296b49-caa5-4885-be3e-168507d8be3a" (UID: "73296b49-caa5-4885-be3e-168507d8be3a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:14:09.363190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.363168    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73296b49-caa5-4885-be3e-168507d8be3a-kube-api-access-d5sgb" (OuterVolumeSpecName: "kube-api-access-d5sgb") pod "73296b49-caa5-4885-be3e-168507d8be3a" (UID: "73296b49-caa5-4885-be3e-168507d8be3a"). InnerVolumeSpecName "kube-api-access-d5sgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:14:09.462069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.462033    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:09.462069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.462063    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5sgb\" (UniqueName: \"kubernetes.io/projected/73296b49-caa5-4885-be3e-168507d8be3a-kube-api-access-d5sgb\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:09.462069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.462074    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73296b49-caa5-4885-be3e-168507d8be3a-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:09.462306 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.462083    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:09.462306 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.462090    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:09.462306 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.462097    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73296b49-caa5-4885-be3e-168507d8be3a-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:09.597483 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.597397    2556 generic.go:356] "Generic (PLEG): container finished" podID="73296b49-caa5-4885-be3e-168507d8be3a" containerID="9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1" exitCode=0
Jul 17 10:14:09.597483 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.597450    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw" event={"ID":"73296b49-caa5-4885-be3e-168507d8be3a","Type":"ContainerDied","Data":"9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1"}
Jul 17 10:14:09.597483 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.597473    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"
Jul 17 10:14:09.597734 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.597495    2556 scope.go:117] "RemoveContainer" containerID="9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1"
Jul 17 10:14:09.597734 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.597484    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw" event={"ID":"73296b49-caa5-4885-be3e-168507d8be3a","Type":"ContainerDied","Data":"da07bdb284ddcd202c4af37c1e47e51f3ecca0365436efee6691511fbd161c65"}
Jul 17 10:14:09.606898 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.606868    2556 scope.go:117] "RemoveContainer" containerID="9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1"
Jul 17 10:14:09.607280 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:09.607250    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1\": container with ID starting with 9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1 not found: ID does not exist" containerID="9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1"
Jul 17 10:14:09.607365 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.607289    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1"} err="failed to get container status \"9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1\": rpc error: code = NotFound desc = could not find container \"9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1\": container with ID starting with 9e4fbcce2fa998f70dfb3eaf92ce30d2cc12f39fe01d06a322306dbf5d0c3fc1 not found: ID does not exist"
Jul 17 10:14:09.619739 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.619683    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"]
Jul 17 10:14:09.624368 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:09.624321    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1c996b/test-model-1c996b-kserve-fc9dc94f-nmtkw"]
Jul 17 10:14:10.495549 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:10.495511    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15cc23ea-caa0-4a20-8289-522550d5c27a" path="/var/lib/kubelet/pods/15cc23ea-caa0-4a20-8289-522550d5c27a/volumes"
Jul 17 10:14:10.495954 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:10.495936    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73296b49-caa5-4885-be3e-168507d8be3a" path="/var/lib/kubelet/pods/73296b49-caa5-4885-be3e-168507d8be3a/volumes"
Jul 17 10:14:15.279672 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:15.279627    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:15.381018 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:15.380975    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:15.664506 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:15.664468    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:14:18.551352 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551311    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"]
Jul 17 10:14:18.551769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551656    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73296b49-caa5-4885-be3e-168507d8be3a" containerName="main"
Jul 17 10:14:18.551769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551673    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="73296b49-caa5-4885-be3e-168507d8be3a" containerName="main"
Jul 17 10:14:18.551769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551691    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15cc23ea-caa0-4a20-8289-522550d5c27a" containerName="main"
Jul 17 10:14:18.551769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551696    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cc23ea-caa0-4a20-8289-522550d5c27a" containerName="main"
Jul 17 10:14:18.551769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551751    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="73296b49-caa5-4885-be3e-168507d8be3a" containerName="main"
Jul 17 10:14:18.551769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.551759    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="15cc23ea-caa0-4a20-8289-522550d5c27a" containerName="main"
Jul 17 10:14:18.556106 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.556088    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:18.558644 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.558621    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:14:18.558780 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.558620    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:14:18.599044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.599008    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"]
Jul 17 10:14:18.637617 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.637573    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12fbd8c-e760-4f28-a99a-9f4fe8434909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m\" (UID: \"f12fbd8c-e760-4f28-a99a-9f4fe8434909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:18.738452 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.738406    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12fbd8c-e760-4f28-a99a-9f4fe8434909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m\" (UID: \"f12fbd8c-e760-4f28-a99a-9f4fe8434909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:18.739094 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.739073    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12fbd8c-e760-4f28-a99a-9f4fe8434909-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m\" (UID: \"f12fbd8c-e760-4f28-a99a-9f4fe8434909\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:18.866500 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:18.866389    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:19.001882 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:19.001858    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"]
Jul 17 10:14:19.004151 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:14:19.004122    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12fbd8c_e760_4f28_a99a_9f4fe8434909.slice/crio-4380cbb23d72e8c0521c2930113b321b78aea07a54de34f601dfbc5ecd8dc65d WatchSource:0}: Error finding container 4380cbb23d72e8c0521c2930113b321b78aea07a54de34f601dfbc5ecd8dc65d: Status 404 returned error can't find the container with id 4380cbb23d72e8c0521c2930113b321b78aea07a54de34f601dfbc5ecd8dc65d
Jul 17 10:14:19.006406 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:19.006389    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:14:19.647050 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:19.647009    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" event={"ID":"f12fbd8c-e760-4f28-a99a-9f4fe8434909","Type":"ContainerStarted","Data":"4380cbb23d72e8c0521c2930113b321b78aea07a54de34f601dfbc5ecd8dc65d"}
Jul 17 10:14:20.358373 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:20.358335    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:20.733976 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:20.733911    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:21.553718 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:21.553686    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:21.654615 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:21.654509    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:21.659698 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:21.659661    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" event={"ID":"f12fbd8c-e760-4f28-a99a-9f4fe8434909","Type":"ContainerStarted","Data":"4097837cd9711556f8e6a9f4e10573fa673736be19e88fe7fce418792e183ae6"}
Jul 17 10:14:21.678622 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:21.678562    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" podStartSLOduration=1.5733827150000002 podStartE2EDuration="3.678543211s" totalImagesPullingTime="2.105160496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:21.677597191 +0000 UTC m=+2845.687721116" watchObservedRunningTime="2026-07-17 10:14:21.678543211 +0000 UTC m=+2845.688667178"
Jul 17 10:14:22.665426 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:22.665388    2556 generic.go:356] "Generic (PLEG): container finished" podID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerID="4097837cd9711556f8e6a9f4e10573fa673736be19e88fe7fce418792e183ae6" exitCode=6
Jul 17 10:14:22.665938 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:22.665454    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" event={"ID":"f12fbd8c-e760-4f28-a99a-9f4fe8434909","Type":"ContainerDied","Data":"4097837cd9711556f8e6a9f4e10573fa673736be19e88fe7fce418792e183ae6"}
Jul 17 10:14:22.665938 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:22.665677    2556 scope.go:117] "RemoveContainer" containerID="4097837cd9711556f8e6a9f4e10573fa673736be19e88fe7fce418792e183ae6"
Jul 17 10:14:23.671102 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:23.671015    2556 generic.go:356] "Generic (PLEG): container finished" podID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerID="15a3858114cda95840370bb8cbd821b3ea7dcb98b06ce9da31e4f0ec15d8717a" exitCode=6
Jul 17 10:14:23.671102 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:23.671068    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" event={"ID":"f12fbd8c-e760-4f28-a99a-9f4fe8434909","Type":"ContainerDied","Data":"15a3858114cda95840370bb8cbd821b3ea7dcb98b06ce9da31e4f0ec15d8717a"}
Jul 17 10:14:23.671647 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:23.671133    2556 scope.go:117] "RemoveContainer" containerID="4097837cd9711556f8e6a9f4e10573fa673736be19e88fe7fce418792e183ae6"
Jul 17 10:14:23.671647 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:23.671329    2556 scope.go:117] "RemoveContainer" containerID="15a3858114cda95840370bb8cbd821b3ea7dcb98b06ce9da31e4f0ec15d8717a"
Jul 17 10:14:23.671647 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:23.671567    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m_odh-ai-gateway-infra(f12fbd8c-e760-4f28-a99a-9f4fe8434909)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909"
Jul 17 10:14:24.677034 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:24.677004    2556 scope.go:117] "RemoveContainer" containerID="15a3858114cda95840370bb8cbd821b3ea7dcb98b06ce9da31e4f0ec15d8717a"
Jul 17 10:14:24.677447 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:24.677182    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m_odh-ai-gateway-infra(f12fbd8c-e760-4f28-a99a-9f4fe8434909)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909"
Jul 17 10:14:25.829495 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:25.829448    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"]
Jul 17 10:14:25.834474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:25.834447    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:25.840731 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:25.840693    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"]
Jul 17 10:14:25.901703 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:25.901660    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc719f1d-d610-4279-b8d7-022de8d0b24a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-grwb8\" (UID: \"fc719f1d-d610-4279-b8d7-022de8d0b24a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:26.002804 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.002758    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc719f1d-d610-4279-b8d7-022de8d0b24a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-grwb8\" (UID: \"fc719f1d-d610-4279-b8d7-022de8d0b24a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:26.003438 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.003413    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc719f1d-d610-4279-b8d7-022de8d0b24a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-grwb8\" (UID: \"fc719f1d-d610-4279-b8d7-022de8d0b24a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:26.147834 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.147785    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:26.295871 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.295843    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"]
Jul 17 10:14:26.298353 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:14:26.298310    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc719f1d_d610_4279_b8d7_022de8d0b24a.slice/crio-3288b7f7fa13ddf8b61ebccd7b29068803b4b8ad748976a01ebeccfd25fbf616 WatchSource:0}: Error finding container 3288b7f7fa13ddf8b61ebccd7b29068803b4b8ad748976a01ebeccfd25fbf616: Status 404 returned error can't find the container with id 3288b7f7fa13ddf8b61ebccd7b29068803b4b8ad748976a01ebeccfd25fbf616
Jul 17 10:14:26.689036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.688991    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerStarted","Data":"7252c24f6bc252212d46f9bc39a17f7f3792f00233ec61c70c1409ea0a6892f9"}
Jul 17 10:14:26.689254 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.689056    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerStarted","Data":"3288b7f7fa13ddf8b61ebccd7b29068803b4b8ad748976a01ebeccfd25fbf616"}
Jul 17 10:14:26.704574 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:26.704514    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" podStartSLOduration=1.7044927319999998 podStartE2EDuration="1.704492732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:26.703061803 +0000 UTC m=+2850.713185737" watchObservedRunningTime="2026-07-17 10:14:26.704492732 +0000 UTC m=+2850.714616635"
Jul 17 10:14:27.694912 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:27.694877    2556 generic.go:356] "Generic (PLEG): container finished" podID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerID="7252c24f6bc252212d46f9bc39a17f7f3792f00233ec61c70c1409ea0a6892f9" exitCode=6
Jul 17 10:14:27.695314 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:27.694925    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerDied","Data":"7252c24f6bc252212d46f9bc39a17f7f3792f00233ec61c70c1409ea0a6892f9"}
Jul 17 10:14:27.695314 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:27.695117    2556 scope.go:117] "RemoveContainer" containerID="7252c24f6bc252212d46f9bc39a17f7f3792f00233ec61c70c1409ea0a6892f9"
Jul 17 10:14:28.701285 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:28.701252    2556 generic.go:356] "Generic (PLEG): container finished" podID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerID="19a42564541ea8006440df0d58a7c71632fb065c017e7056b0bedd07956415a6" exitCode=6
Jul 17 10:14:28.701747 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:28.701343    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerDied","Data":"19a42564541ea8006440df0d58a7c71632fb065c017e7056b0bedd07956415a6"}
Jul 17 10:14:28.701747 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:28.701413    2556 scope.go:117] "RemoveContainer" containerID="7252c24f6bc252212d46f9bc39a17f7f3792f00233ec61c70c1409ea0a6892f9"
Jul 17 10:14:28.701747 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:28.701645    2556 scope.go:117] "RemoveContainer" containerID="19a42564541ea8006440df0d58a7c71632fb065c017e7056b0bedd07956415a6"
Jul 17 10:14:28.701909 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:28.701888    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-grwb8_odh-ai-gateway-infra(fc719f1d-d610-4279-b8d7-022de8d0b24a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a"
Jul 17 10:14:29.707061 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:29.707029    2556 scope.go:117] "RemoveContainer" containerID="19a42564541ea8006440df0d58a7c71632fb065c017e7056b0bedd07956415a6"
Jul 17 10:14:29.707452 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:29.707211    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-grwb8_odh-ai-gateway-infra(fc719f1d-d610-4279-b8d7-022de8d0b24a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a"
Jul 17 10:14:31.330190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:31.330146    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:34.768494 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:34.768439    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:14:35.965297 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:35.965258    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:37.448058 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.448020    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"]
Jul 17 10:14:37.451491 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.451467    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:37.458529 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.458451    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"]
Jul 17 10:14:37.596460 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.596397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc\" (UID: \"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:37.697597 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.697550    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc\" (UID: \"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:37.698309 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.698261    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc\" (UID: \"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:37.762711 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.762672    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:37.900169 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:37.900142    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"]
Jul 17 10:14:37.902945 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:14:37.902912    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a9d02f_1326_4b7a_b8cb_2f13987d80dc.slice/crio-b70e556aa845cd48dbbfab8489104edf980bcc49a1d76d597253de36bd2fcef3 WatchSource:0}: Error finding container b70e556aa845cd48dbbfab8489104edf980bcc49a1d76d597253de36bd2fcef3: Status 404 returned error can't find the container with id b70e556aa845cd48dbbfab8489104edf980bcc49a1d76d597253de36bd2fcef3
Jul 17 10:14:38.752430 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:38.752392    2556 generic.go:356] "Generic (PLEG): container finished" podID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerID="cfa83c0f3f576f1b63d85e0d22af925447006bdf90620046086a3d34e0c981ac" exitCode=6
Jul 17 10:14:38.752799 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:38.752490    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" event={"ID":"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc","Type":"ContainerDied","Data":"cfa83c0f3f576f1b63d85e0d22af925447006bdf90620046086a3d34e0c981ac"}
Jul 17 10:14:38.752799 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:38.752544    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" event={"ID":"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc","Type":"ContainerStarted","Data":"b70e556aa845cd48dbbfab8489104edf980bcc49a1d76d597253de36bd2fcef3"}
Jul 17 10:14:38.752799 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:38.752669    2556 scope.go:117] "RemoveContainer" containerID="cfa83c0f3f576f1b63d85e0d22af925447006bdf90620046086a3d34e0c981ac"
Jul 17 10:14:39.490076 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:39.490050    2556 scope.go:117] "RemoveContainer" containerID="15a3858114cda95840370bb8cbd821b3ea7dcb98b06ce9da31e4f0ec15d8717a"
Jul 17 10:14:39.758353 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:39.758272    2556 generic.go:356] "Generic (PLEG): container finished" podID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerID="10255dc48d58fbed59de57f90fe404d5eebff41177d458051f3fe62d17522640" exitCode=6
Jul 17 10:14:39.758353 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:39.758327    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" event={"ID":"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc","Type":"ContainerDied","Data":"10255dc48d58fbed59de57f90fe404d5eebff41177d458051f3fe62d17522640"}
Jul 17 10:14:39.758794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:39.758385    2556 scope.go:117] "RemoveContainer" containerID="cfa83c0f3f576f1b63d85e0d22af925447006bdf90620046086a3d34e0c981ac"
Jul 17 10:14:39.758794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:39.758554    2556 scope.go:117] "RemoveContainer" containerID="10255dc48d58fbed59de57f90fe404d5eebff41177d458051f3fe62d17522640"
Jul 17 10:14:39.758794 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:39.758767    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc_odh-ai-gateway-infra(c8a9d02f-1326-4b7a-b8cb-2f13987d80dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc"
Jul 17 10:14:40.764267 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:40.764233    2556 generic.go:356] "Generic (PLEG): container finished" podID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerID="ce38f028112cf8c9a8793ba209ad32030d82e0d89f2f85aa4bc6eab1e88a078b" exitCode=6
Jul 17 10:14:40.764737 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:40.764307    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" event={"ID":"f12fbd8c-e760-4f28-a99a-9f4fe8434909","Type":"ContainerDied","Data":"ce38f028112cf8c9a8793ba209ad32030d82e0d89f2f85aa4bc6eab1e88a078b"}
Jul 17 10:14:40.764737 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:40.764374    2556 scope.go:117] "RemoveContainer" containerID="15a3858114cda95840370bb8cbd821b3ea7dcb98b06ce9da31e4f0ec15d8717a"
Jul 17 10:14:40.764737 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:40.764710    2556 scope.go:117] "RemoveContainer" containerID="ce38f028112cf8c9a8793ba209ad32030d82e0d89f2f85aa4bc6eab1e88a078b"
Jul 17 10:14:40.764975 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:40.764949    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m_odh-ai-gateway-infra(f12fbd8c-e760-4f28-a99a-9f4fe8434909)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909"
Jul 17 10:14:40.766221 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:40.766204    2556 scope.go:117] "RemoveContainer" containerID="10255dc48d58fbed59de57f90fe404d5eebff41177d458051f3fe62d17522640"
Jul 17 10:14:40.766399 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:40.766381    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc_odh-ai-gateway-infra(c8a9d02f-1326-4b7a-b8cb-2f13987d80dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc"
Jul 17 10:14:41.790456 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:41.790416    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"]
Jul 17 10:14:41.824695 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:41.824659    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:41.891350 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:41.891312    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:41.934599 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:41.934569    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:42.033754 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.033716    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12fbd8c-e760-4f28-a99a-9f4fe8434909-maas-api-service-ca\") pod \"f12fbd8c-e760-4f28-a99a-9f4fe8434909\" (UID: \"f12fbd8c-e760-4f28-a99a-9f4fe8434909\") "
Jul 17 10:14:42.034156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.034133    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f12fbd8c-e760-4f28-a99a-9f4fe8434909-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f12fbd8c-e760-4f28-a99a-9f4fe8434909" (UID: "f12fbd8c-e760-4f28-a99a-9f4fe8434909"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:42.135044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.135012    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f12fbd8c-e760-4f28-a99a-9f4fe8434909-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:42.778848 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.778736    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"
Jul 17 10:14:42.778848 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.778772    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m" event={"ID":"f12fbd8c-e760-4f28-a99a-9f4fe8434909","Type":"ContainerDied","Data":"4380cbb23d72e8c0521c2930113b321b78aea07a54de34f601dfbc5ecd8dc65d"}
Jul 17 10:14:42.779065 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.778885    2556 scope.go:117] "RemoveContainer" containerID="ce38f028112cf8c9a8793ba209ad32030d82e0d89f2f85aa4bc6eab1e88a078b"
Jul 17 10:14:42.795633 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.795604    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"]
Jul 17 10:14:42.806136 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:42.806098    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x2p6m"]
Jul 17 10:14:44.490235 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:44.490197    2556 scope.go:117] "RemoveContainer" containerID="19a42564541ea8006440df0d58a7c71632fb065c017e7056b0bedd07956415a6"
Jul 17 10:14:44.495343 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:44.495308    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" path="/var/lib/kubelet/pods/f12fbd8c-e760-4f28-a99a-9f4fe8434909/volumes"
Jul 17 10:14:44.790238 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:44.790147    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerStarted","Data":"bc2f2718199e16c8928f4f2f64077ee384efc704853cb5a9302982c3b00b7c55"}
Jul 17 10:14:45.798937 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:45.798904    2556 generic.go:356] "Generic (PLEG): container finished" podID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerID="bc2f2718199e16c8928f4f2f64077ee384efc704853cb5a9302982c3b00b7c55" exitCode=6
Jul 17 10:14:45.799325 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:45.798979    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerDied","Data":"bc2f2718199e16c8928f4f2f64077ee384efc704853cb5a9302982c3b00b7c55"}
Jul 17 10:14:45.799325 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:45.799044    2556 scope.go:117] "RemoveContainer" containerID="19a42564541ea8006440df0d58a7c71632fb065c017e7056b0bedd07956415a6"
Jul 17 10:14:45.799427 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:45.799386    2556 scope.go:117] "RemoveContainer" containerID="bc2f2718199e16c8928f4f2f64077ee384efc704853cb5a9302982c3b00b7c55"
Jul 17 10:14:45.799617 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:45.799601    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-grwb8_odh-ai-gateway-infra(fc719f1d-d610-4279-b8d7-022de8d0b24a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a"
Jul 17 10:14:45.813738 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:45.813709    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"]
Jul 17 10:14:46.427310 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427277    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"]
Jul 17 10:14:46.427582 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427570    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.427637 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427584    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.427637 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427591    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.427637 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427596    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.427731 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427655    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.427731 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427665    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.427731 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.427672    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:14:46.431791 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.431771    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:14:46.438178 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.438140    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"]
Jul 17 10:14:46.576546 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.576503    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ed1512f-6faa-4a00-9623-bc5454b3ddb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f\" (UID: \"3ed1512f-6faa-4a00-9623-bc5454b3ddb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:14:46.677651 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.677529    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ed1512f-6faa-4a00-9623-bc5454b3ddb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f\" (UID: \"3ed1512f-6faa-4a00-9623-bc5454b3ddb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:14:46.678217 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.678195    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ed1512f-6faa-4a00-9623-bc5454b3ddb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f\" (UID: \"3ed1512f-6faa-4a00-9623-bc5454b3ddb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:14:46.720485 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.720436    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:46.745313 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.745281    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:14:46.899192 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.899159    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"]
Jul 17 10:14:46.901733 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:14:46.901697    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed1512f_6faa_4a00_9623_bc5454b3ddb7.slice/crio-398ffb6132644e62d368877de0af296c064f0f472f2b50b1acb9385f70fa616a WatchSource:0}: Error finding container 398ffb6132644e62d368877de0af296c064f0f472f2b50b1acb9385f70fa616a: Status 404 returned error can't find the container with id 398ffb6132644e62d368877de0af296c064f0f472f2b50b1acb9385f70fa616a
Jul 17 10:14:46.913862 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.913817    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:46.953215 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:46.953183    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:47.080678 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.080639    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc719f1d-d610-4279-b8d7-022de8d0b24a-maas-api-service-ca\") pod \"fc719f1d-d610-4279-b8d7-022de8d0b24a\" (UID: \"fc719f1d-d610-4279-b8d7-022de8d0b24a\") "
Jul 17 10:14:47.081011 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.080990    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc719f1d-d610-4279-b8d7-022de8d0b24a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc719f1d-d610-4279-b8d7-022de8d0b24a" (UID: "fc719f1d-d610-4279-b8d7-022de8d0b24a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:47.181202 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.181167    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc719f1d-d610-4279-b8d7-022de8d0b24a-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:47.270350 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.270307    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:47.572238 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.572201    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:47.814638 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.814599    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"
Jul 17 10:14:47.814863 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.814636    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8" event={"ID":"fc719f1d-d610-4279-b8d7-022de8d0b24a","Type":"ContainerDied","Data":"3288b7f7fa13ddf8b61ebccd7b29068803b4b8ad748976a01ebeccfd25fbf616"}
Jul 17 10:14:47.814863 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.814702    2556 scope.go:117] "RemoveContainer" containerID="bc2f2718199e16c8928f4f2f64077ee384efc704853cb5a9302982c3b00b7c55"
Jul 17 10:14:47.816271 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.816243    2556 generic.go:356] "Generic (PLEG): container finished" podID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerID="3f208c4c12ac2b5f94be2fb4f7812ca574c348548e8a462c496929cffde52033" exitCode=6
Jul 17 10:14:47.816397 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.816282    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" event={"ID":"3ed1512f-6faa-4a00-9623-bc5454b3ddb7","Type":"ContainerDied","Data":"3f208c4c12ac2b5f94be2fb4f7812ca574c348548e8a462c496929cffde52033"}
Jul 17 10:14:47.816397 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.816332    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" event={"ID":"3ed1512f-6faa-4a00-9623-bc5454b3ddb7","Type":"ContainerStarted","Data":"398ffb6132644e62d368877de0af296c064f0f472f2b50b1acb9385f70fa616a"}
Jul 17 10:14:47.816541 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.816528    2556 scope.go:117] "RemoveContainer" containerID="3f208c4c12ac2b5f94be2fb4f7812ca574c348548e8a462c496929cffde52033"
Jul 17 10:14:47.862005 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.861969    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"]
Jul 17 10:14:47.862375 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.862354    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:47.866928 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:47.866902    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-grwb8"]
Jul 17 10:14:48.495642 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:48.495600    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" path="/var/lib/kubelet/pods/fc719f1d-d610-4279-b8d7-022de8d0b24a/volumes"
Jul 17 10:14:48.604726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:48.604689    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:48.823636 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:48.823545    2556 generic.go:356] "Generic (PLEG): container finished" podID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerID="31a14c9f45c7e4887283621902866f5b35f003c7e23811babffb168d906e17be" exitCode=6
Jul 17 10:14:48.823636 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:48.823592    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" event={"ID":"3ed1512f-6faa-4a00-9623-bc5454b3ddb7","Type":"ContainerDied","Data":"31a14c9f45c7e4887283621902866f5b35f003c7e23811babffb168d906e17be"}
Jul 17 10:14:48.823636 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:48.823637    2556 scope.go:117] "RemoveContainer" containerID="3f208c4c12ac2b5f94be2fb4f7812ca574c348548e8a462c496929cffde52033"
Jul 17 10:14:48.823922 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:48.823881    2556 scope.go:117] "RemoveContainer" containerID="31a14c9f45c7e4887283621902866f5b35f003c7e23811babffb168d906e17be"
Jul 17 10:14:48.824091 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:48.824068    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f_odh-ai-gateway-infra(3ed1512f-6faa-4a00-9623-bc5454b3ddb7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7"
Jul 17 10:14:49.831409 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:49.831361    2556 scope.go:117] "RemoveContainer" containerID="31a14c9f45c7e4887283621902866f5b35f003c7e23811babffb168d906e17be"
Jul 17 10:14:49.832035 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:49.831750    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f_odh-ai-gateway-infra(3ed1512f-6faa-4a00-9623-bc5454b3ddb7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7"
Jul 17 10:14:51.083373 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:51.083338    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:51.178981 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:51.178948    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:51.254559 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:51.254514    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:51.389424 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:51.389388    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:51.727191 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:51.727091    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:51.831740 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:51.831708    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:54.490307 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:54.490266    2556 scope.go:117] "RemoveContainer" containerID="10255dc48d58fbed59de57f90fe404d5eebff41177d458051f3fe62d17522640"
Jul 17 10:14:54.857991 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:54.857948    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" event={"ID":"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc","Type":"ContainerStarted","Data":"1f8c1df6e14f07c7ab6b8275d62e94732ee7369d9dae249c1ce5050e6cd268c6"}
Jul 17 10:14:54.874899 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:54.874840    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" podStartSLOduration=17.874802698 podStartE2EDuration="17.874802698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:54.873355616 +0000 UTC m=+2878.883479542" watchObservedRunningTime="2026-07-17 10:14:54.874802698 +0000 UTC m=+2878.884926686"
Jul 17 10:14:55.062776 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.062686    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:14:55.457907 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.457874    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:55.863433 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.863343    2556 generic.go:356] "Generic (PLEG): container finished" podID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerID="1f8c1df6e14f07c7ab6b8275d62e94732ee7369d9dae249c1ce5050e6cd268c6" exitCode=6
Jul 17 10:14:55.863889 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.863425    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" event={"ID":"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc","Type":"ContainerDied","Data":"1f8c1df6e14f07c7ab6b8275d62e94732ee7369d9dae249c1ce5050e6cd268c6"}
Jul 17 10:14:55.863889 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.863501    2556 scope.go:117] "RemoveContainer" containerID="10255dc48d58fbed59de57f90fe404d5eebff41177d458051f3fe62d17522640"
Jul 17 10:14:55.863889 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.863775    2556 scope.go:117] "RemoveContainer" containerID="1f8c1df6e14f07c7ab6b8275d62e94732ee7369d9dae249c1ce5050e6cd268c6"
Jul 17 10:14:55.864007 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:14:55.863970    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc_odh-ai-gateway-infra(c8a9d02f-1326-4b7a-b8cb-2f13987d80dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc"
Jul 17 10:14:55.886574 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.886539    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"]
Jul 17 10:14:55.943636 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:55.943602    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:56.257529 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:56.257479    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:56.503321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:56.503289    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:57.005198 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.005171    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:57.166294 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.166259    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc-maas-api-service-ca\") pod \"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc\" (UID: \"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc\") "
Jul 17 10:14:57.166688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.166664    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" (UID: "c8a9d02f-1326-4b7a-b8cb-2f13987d80dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:57.267944 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.267852    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:14:57.874507 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.874477    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"
Jul 17 10:14:57.874701 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.874478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc" event={"ID":"c8a9d02f-1326-4b7a-b8cb-2f13987d80dc","Type":"ContainerDied","Data":"b70e556aa845cd48dbbfab8489104edf980bcc49a1d76d597253de36bd2fcef3"}
Jul 17 10:14:57.874701 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.874622    2556 scope.go:117] "RemoveContainer" containerID="1f8c1df6e14f07c7ab6b8275d62e94732ee7369d9dae249c1ce5050e6cd268c6"
Jul 17 10:14:57.897866 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.897807    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"]
Jul 17 10:14:57.910096 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:57.904676    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jkqdc"]
Jul 17 10:14:58.494426 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:58.494391    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" path="/var/lib/kubelet/pods/c8a9d02f-1326-4b7a-b8cb-2f13987d80dc/volumes"
Jul 17 10:14:59.241388 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:59.241340    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:59.322567 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:59.322524    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:59.395489 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:59.395443    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:59.549319 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:59.549219    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:59.890410 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:59.890373    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:14:59.994082 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:14:59.994044    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:00.133891 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.133850    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"]
Jul 17 10:15:00.134226 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134208    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134230    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12fbd8c-e760-4f28-a99a-9f4fe8434909" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134243    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134252    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134265    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134274    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134289    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134298    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134316    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134324    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134405    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.134750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134416    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134429    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134439    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.134750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134446    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.134750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.134458    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.137355 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.137331    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"
Jul 17 10:15:00.142922 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.142864    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"]
Jul 17 10:15:00.143245 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.143226    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.143345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.143247    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc719f1d-d610-4279-b8d7-022de8d0b24a" containerName="revoke-keys"
Jul 17 10:15:00.143345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.143274    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.143345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.143282    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8a9d02f-1326-4b7a-b8cb-2f13987d80dc" containerName="revoke-keys"
Jul 17 10:15:00.145283 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.145261    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"]
Jul 17 10:15:00.145392 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.145367    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"
Jul 17 10:15:00.147153 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.147136    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"
Jul 17 10:15:00.147273 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.147248    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"
Jul 17 10:15:00.154149 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.154118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"]
Jul 17 10:15:00.157143 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.157113    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"]
Jul 17 10:15:00.159979 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.159950    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"]
Jul 17 10:15:00.180890 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.180706    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"
Jul 17 10:15:00.190146 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.190110    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"
Jul 17 10:15:00.333248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.332351    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"]
Jul 17 10:15:00.391870 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.391594    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"]
Jul 17 10:15:00.395078 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:00.395010    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod937e54b9_db3e_4c14_b98f_e1d650c5e3cf.slice/crio-843e99b6bdb2997f781fc49ebef04649ca6d01a002ff0910492ef624d31dd1cd WatchSource:0}: Error finding container 843e99b6bdb2997f781fc49ebef04649ca6d01a002ff0910492ef624d31dd1cd: Status 404 returned error can't find the container with id 843e99b6bdb2997f781fc49ebef04649ca6d01a002ff0910492ef624d31dd1cd
Jul 17 10:15:00.418529 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.418494    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"]
Jul 17 10:15:00.431253 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:00.431218    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979c9f59_4ace_4a90_ba49_bc17271f0994.slice/crio-c906bf391df9b03778f5997e7c262959fd4ab4503879fc7e89bdcf3477f232b5 WatchSource:0}: Error finding container c906bf391df9b03778f5997e7c262959fd4ab4503879fc7e89bdcf3477f232b5: Status 404 returned error can't find the container with id c906bf391df9b03778f5997e7c262959fd4ab4503879fc7e89bdcf3477f232b5
Jul 17 10:15:00.490319 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.490291    2556 scope.go:117] "RemoveContainer" containerID="31a14c9f45c7e4887283621902866f5b35f003c7e23811babffb168d906e17be"
Jul 17 10:15:00.891741 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.891706    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerStarted","Data":"e8b2dd32c135398c01d188dd071c26e7751a10d98e477b513b7ef3ca94f5413c"}
Jul 17 10:15:00.892221 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.891768    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerStarted","Data":"c1dad0b57366553b37bee4517be86144a051f5664678c36ef12098daddf43d42"}
Jul 17 10:15:00.893940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.893903    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" event={"ID":"3ed1512f-6faa-4a00-9623-bc5454b3ddb7","Type":"ContainerStarted","Data":"5a12f2700fbdadb9c0bc4e3c0aea8ed54dc20477cfb5e5467708eaf21ebf7e17"}
Jul 17 10:15:00.895496 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.895473    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc" event={"ID":"979c9f59-4ace-4a90-ba49-bc17271f0994","Type":"ContainerStarted","Data":"ff602cc4f8868cbafaf24de3b9800cc6842e09a6a3355ec25c7c5dfe2f459012"}
Jul 17 10:15:00.895639 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.895517    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc" event={"ID":"979c9f59-4ace-4a90-ba49-bc17271f0994","Type":"ContainerStarted","Data":"c906bf391df9b03778f5997e7c262959fd4ab4503879fc7e89bdcf3477f232b5"}
Jul 17 10:15:00.897098 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.897074    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2" event={"ID":"937e54b9-db3e-4c14-b98f-e1d650c5e3cf","Type":"ContainerStarted","Data":"bb48c5116b92cd971694d686c44edad0b3c3f68467d462b76c1216c698861394"}
Jul 17 10:15:00.897216 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.897111    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2" event={"ID":"937e54b9-db3e-4c14-b98f-e1d650c5e3cf","Type":"ContainerStarted","Data":"843e99b6bdb2997f781fc49ebef04649ca6d01a002ff0910492ef624d31dd1cd"}
Jul 17 10:15:00.907742 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.907684    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" podStartSLOduration=0.907666052 podStartE2EDuration="907.666052ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:00.906104617 +0000 UTC m=+2884.916228541" watchObservedRunningTime="2026-07-17 10:15:00.907666052 +0000 UTC m=+2884.917789975"
Jul 17 10:15:00.921396 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.921334    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2" podStartSLOduration=0.92131453 podStartE2EDuration="921.31453ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:00.920206646 +0000 UTC m=+2884.930330573" watchObservedRunningTime="2026-07-17 10:15:00.92131453 +0000 UTC m=+2884.931438454"
Jul 17 10:15:00.934533 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.934471    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc" podStartSLOduration=0.934454817 podStartE2EDuration="934.454817ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:00.933296429 +0000 UTC m=+2884.943420346" watchObservedRunningTime="2026-07-17 10:15:00.934454817 +0000 UTC m=+2884.944578741"
Jul 17 10:15:00.949073 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:00.948966    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" podStartSLOduration=14.948950678 podStartE2EDuration="14.948950678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:00.947760717 +0000 UTC m=+2884.957884655" watchObservedRunningTime="2026-07-17 10:15:00.948950678 +0000 UTC m=+2884.959074601"
Jul 17 10:15:01.903559 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.903529    2556 generic.go:356] "Generic (PLEG): container finished" podID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerID="e8b2dd32c135398c01d188dd071c26e7751a10d98e477b513b7ef3ca94f5413c" exitCode=7
Jul 17 10:15:01.904035 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.903604    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerDied","Data":"e8b2dd32c135398c01d188dd071c26e7751a10d98e477b513b7ef3ca94f5413c"}
Jul 17 10:15:01.904035 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.903760    2556 scope.go:117] "RemoveContainer" containerID="e8b2dd32c135398c01d188dd071c26e7751a10d98e477b513b7ef3ca94f5413c"
Jul 17 10:15:01.905744 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.905716    2556 generic.go:356] "Generic (PLEG): container finished" podID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerID="5a12f2700fbdadb9c0bc4e3c0aea8ed54dc20477cfb5e5467708eaf21ebf7e17" exitCode=6
Jul 17 10:15:01.905886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.905745    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" event={"ID":"3ed1512f-6faa-4a00-9623-bc5454b3ddb7","Type":"ContainerDied","Data":"5a12f2700fbdadb9c0bc4e3c0aea8ed54dc20477cfb5e5467708eaf21ebf7e17"}
Jul 17 10:15:01.905886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.905797    2556 scope.go:117] "RemoveContainer" containerID="31a14c9f45c7e4887283621902866f5b35f003c7e23811babffb168d906e17be"
Jul 17 10:15:01.906080 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.906062    2556 scope.go:117] "RemoveContainer" containerID="5a12f2700fbdadb9c0bc4e3c0aea8ed54dc20477cfb5e5467708eaf21ebf7e17"
Jul 17 10:15:01.906281 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:01.906255    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f_odh-ai-gateway-infra(3ed1512f-6faa-4a00-9623-bc5454b3ddb7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7"
Jul 17 10:15:01.907476 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.907453    2556 generic.go:356] "Generic (PLEG): container finished" podID="979c9f59-4ace-4a90-ba49-bc17271f0994" containerID="ff602cc4f8868cbafaf24de3b9800cc6842e09a6a3355ec25c7c5dfe2f459012" exitCode=0
Jul 17 10:15:01.907568 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.907519    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc" event={"ID":"979c9f59-4ace-4a90-ba49-bc17271f0994","Type":"ContainerDied","Data":"ff602cc4f8868cbafaf24de3b9800cc6842e09a6a3355ec25c7c5dfe2f459012"}
Jul 17 10:15:01.909203 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.909178    2556 generic.go:356] "Generic (PLEG): container finished" podID="937e54b9-db3e-4c14-b98f-e1d650c5e3cf" containerID="bb48c5116b92cd971694d686c44edad0b3c3f68467d462b76c1216c698861394" exitCode=0
Jul 17 10:15:01.909297 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.909261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2" event={"ID":"937e54b9-db3e-4c14-b98f-e1d650c5e3cf","Type":"ContainerDied","Data":"bb48c5116b92cd971694d686c44edad0b3c3f68467d462b76c1216c698861394"}
Jul 17 10:15:01.958151 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:01.958113    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"]
Jul 17 10:15:02.916936 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:02.916889    2556 generic.go:356] "Generic (PLEG): container finished" podID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerID="9cbedb2fadff807bbc46c16a4533b1540027f75461bab50e4dc01a8df358fc8b" exitCode=7
Jul 17 10:15:02.917387 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:02.917014    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerDied","Data":"9cbedb2fadff807bbc46c16a4533b1540027f75461bab50e4dc01a8df358fc8b"}
Jul 17 10:15:02.917387 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:02.917079    2556 scope.go:117] "RemoveContainer" containerID="e8b2dd32c135398c01d188dd071c26e7751a10d98e477b513b7ef3ca94f5413c"
Jul 17 10:15:02.917490 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:02.917442    2556 scope.go:117] "RemoveContainer" containerID="9cbedb2fadff807bbc46c16a4533b1540027f75461bab50e4dc01a8df358fc8b"
Jul 17 10:15:02.917748 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:02.917713    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5_odh-ai-gateway-infra(5730ca08-70a5-48e7-99ff-17cd7acb17ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce"
Jul 17 10:15:03.075944 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.075916    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:15:03.106186 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.106162    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"
Jul 17 10:15:03.109549 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.109527    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"
Jul 17 10:15:03.112845 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.112811    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ed1512f-6faa-4a00-9623-bc5454b3ddb7-maas-api-service-ca\") pod \"3ed1512f-6faa-4a00-9623-bc5454b3ddb7\" (UID: \"3ed1512f-6faa-4a00-9623-bc5454b3ddb7\") "
Jul 17 10:15:03.113136 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.113118    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ed1512f-6faa-4a00-9623-bc5454b3ddb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ed1512f-6faa-4a00-9623-bc5454b3ddb7" (UID: "3ed1512f-6faa-4a00-9623-bc5454b3ddb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:03.214414 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.214332    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ed1512f-6faa-4a00-9623-bc5454b3ddb7-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:03.791723 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.791685    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:03.926498 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.926470    2556 scope.go:117] "RemoveContainer" containerID="9cbedb2fadff807bbc46c16a4533b1540027f75461bab50e4dc01a8df358fc8b"
Jul 17 10:15:03.926976 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:03.926718    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5_odh-ai-gateway-infra(5730ca08-70a5-48e7-99ff-17cd7acb17ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce"
Jul 17 10:15:03.927543 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.927522    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"
Jul 17 10:15:03.927634 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.927523    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f" event={"ID":"3ed1512f-6faa-4a00-9623-bc5454b3ddb7","Type":"ContainerDied","Data":"398ffb6132644e62d368877de0af296c064f0f472f2b50b1acb9385f70fa616a"}
Jul 17 10:15:03.927691 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.927660    2556 scope.go:117] "RemoveContainer" containerID="5a12f2700fbdadb9c0bc4e3c0aea8ed54dc20477cfb5e5467708eaf21ebf7e17"
Jul 17 10:15:03.928947 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.928853    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc"
Jul 17 10:15:03.929010 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.928849    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-zm7nc" event={"ID":"979c9f59-4ace-4a90-ba49-bc17271f0994","Type":"ContainerDied","Data":"c906bf391df9b03778f5997e7c262959fd4ab4503879fc7e89bdcf3477f232b5"}
Jul 17 10:15:03.929010 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.928996    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c906bf391df9b03778f5997e7c262959fd4ab4503879fc7e89bdcf3477f232b5"
Jul 17 10:15:03.930258 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.930241    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"
Jul 17 10:15:03.930389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.930274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2" event={"ID":"937e54b9-db3e-4c14-b98f-e1d650c5e3cf","Type":"ContainerDied","Data":"843e99b6bdb2997f781fc49ebef04649ca6d01a002ff0910492ef624d31dd1cd"}
Jul 17 10:15:03.930389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.930322    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="843e99b6bdb2997f781fc49ebef04649ca6d01a002ff0910492ef624d31dd1cd"
Jul 17 10:15:03.957437 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.957404    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"]
Jul 17 10:15:03.960989 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:03.960957    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tb85f"]
Jul 17 10:15:04.494993 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:04.494961    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" path="/var/lib/kubelet/pods/3ed1512f-6faa-4a00-9623-bc5454b3ddb7/volumes"
Jul 17 10:15:07.767365 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:07.767317    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:15:11.792024 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.791982    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"]
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792287    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="937e54b9-db3e-4c14-b98f-e1d650c5e3cf" containerName="cleanup"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792299    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="937e54b9-db3e-4c14-b98f-e1d650c5e3cf" containerName="cleanup"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792317    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="979c9f59-4ace-4a90-ba49-bc17271f0994" containerName="cleanup"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792324    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="979c9f59-4ace-4a90-ba49-bc17271f0994" containerName="cleanup"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792330    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792337    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792349    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792354    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792359    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792364    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792700 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792409    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792700 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792418    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="979c9f59-4ace-4a90-ba49-bc17271f0994" containerName="cleanup"
Jul 17 10:15:11.792700 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792425    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="937e54b9-db3e-4c14-b98f-e1d650c5e3cf" containerName="cleanup"
Jul 17 10:15:11.792700 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792431    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.792700 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.792438    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed1512f-6faa-4a00-9623-bc5454b3ddb7" containerName="revoke-keys"
Jul 17 10:15:11.794786 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.794768    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:11.797099 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.797063    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:15:11.803890 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.803865    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"]
Jul 17 10:15:11.884005 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.883958    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e9e9264-4b59-4fc0-ba10-c96f05ef2494-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd\" (UID: \"2e9e9264-4b59-4fc0-ba10-c96f05ef2494\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:11.985315 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.985253    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e9e9264-4b59-4fc0-ba10-c96f05ef2494-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd\" (UID: \"2e9e9264-4b59-4fc0-ba10-c96f05ef2494\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:11.986001 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:11.985978    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e9e9264-4b59-4fc0-ba10-c96f05ef2494-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd\" (UID: \"2e9e9264-4b59-4fc0-ba10-c96f05ef2494\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:12.107046 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.106952    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:12.237619 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.237451    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"]
Jul 17 10:15:12.240732 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:12.240701    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e9e9264_4b59_4fc0_ba10_c96f05ef2494.slice/crio-0f72546c3319a693613d71e49451d818be99b385f2731b2487d4483d978fa905 WatchSource:0}: Error finding container 0f72546c3319a693613d71e49451d818be99b385f2731b2487d4483d978fa905: Status 404 returned error can't find the container with id 0f72546c3319a693613d71e49451d818be99b385f2731b2487d4483d978fa905
Jul 17 10:15:12.335531 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.335489    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:12.437404 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.437360    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:12.975280 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.975242    2556 generic.go:356] "Generic (PLEG): container finished" podID="2e9e9264-4b59-4fc0-ba10-c96f05ef2494" containerID="b2048411f44ce04e6fdca3777f549ec1aeb6533e9c9218fb5c82acea8ce8cae3" exitCode=0
Jul 17 10:15:12.975672 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.975284    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd" event={"ID":"2e9e9264-4b59-4fc0-ba10-c96f05ef2494","Type":"ContainerDied","Data":"b2048411f44ce04e6fdca3777f549ec1aeb6533e9c9218fb5c82acea8ce8cae3"}
Jul 17 10:15:12.975672 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:12.975347    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd" event={"ID":"2e9e9264-4b59-4fc0-ba10-c96f05ef2494","Type":"ContainerStarted","Data":"0f72546c3319a693613d71e49451d818be99b385f2731b2487d4483d978fa905"}
Jul 17 10:15:14.108837 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.108792    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:14.204445 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.204389    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e9e9264-4b59-4fc0-ba10-c96f05ef2494-maas-api-service-ca\") pod \"2e9e9264-4b59-4fc0-ba10-c96f05ef2494\" (UID: \"2e9e9264-4b59-4fc0-ba10-c96f05ef2494\") "
Jul 17 10:15:14.204787 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.204762    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e9e9264-4b59-4fc0-ba10-c96f05ef2494-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e9e9264-4b59-4fc0-ba10-c96f05ef2494" (UID: "2e9e9264-4b59-4fc0-ba10-c96f05ef2494"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:14.305755 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.305662    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e9e9264-4b59-4fc0-ba10-c96f05ef2494-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:14.987090 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.987057    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd"
Jul 17 10:15:14.987267 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.987061    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd" event={"ID":"2e9e9264-4b59-4fc0-ba10-c96f05ef2494","Type":"ContainerDied","Data":"0f72546c3319a693613d71e49451d818be99b385f2731b2487d4483d978fa905"}
Jul 17 10:15:14.987267 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:14.987183    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f72546c3319a693613d71e49451d818be99b385f2731b2487d4483d978fa905"
Jul 17 10:15:16.492975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:16.492880    2556 scope.go:117] "RemoveContainer" containerID="9cbedb2fadff807bbc46c16a4533b1540027f75461bab50e4dc01a8df358fc8b"
Jul 17 10:15:16.998351 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:16.998315    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerStarted","Data":"191edde92745ad30ade2f58d834d6ab3dd87d3bdb3888d3494cab5734f02e10f"}
Jul 17 10:15:18.005196 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.005162    2556 generic.go:356] "Generic (PLEG): container finished" podID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerID="191edde92745ad30ade2f58d834d6ab3dd87d3bdb3888d3494cab5734f02e10f" exitCode=0
Jul 17 10:15:18.005584 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.005223    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerDied","Data":"191edde92745ad30ade2f58d834d6ab3dd87d3bdb3888d3494cab5734f02e10f"}
Jul 17 10:15:18.005584 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.005273    2556 scope.go:117] "RemoveContainer" containerID="9cbedb2fadff807bbc46c16a4533b1540027f75461bab50e4dc01a8df358fc8b"
Jul 17 10:15:18.028749 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.028712    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"]
Jul 17 10:15:18.597899 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.597859    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"]
Jul 17 10:15:18.598217 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.598204    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e9e9264-4b59-4fc0-ba10-c96f05ef2494" containerName="revoke-keys"
Jul 17 10:15:18.598267 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.598219    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e9e9264-4b59-4fc0-ba10-c96f05ef2494" containerName="revoke-keys"
Jul 17 10:15:18.598302 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.598286    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e9e9264-4b59-4fc0-ba10-c96f05ef2494" containerName="revoke-keys"
Jul 17 10:15:18.601010 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.600990    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:18.603375 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.603355    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:15:18.608399 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.608379    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"]
Jul 17 10:15:18.641188 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.641112    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-6spln\" (UID: \"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:18.741895 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.741803    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-6spln\" (UID: \"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:18.742497 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.742472    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-6spln\" (UID: \"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:18.912446 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:18.912408    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:19.051251 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:19.051216    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"]
Jul 17 10:15:19.055772 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:19.055732    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43f49b6_0b87_4d17_bb0c_aa67eec8a3ae.slice/crio-8afec5a8c09e9fad1d9971bf31476ed0119d5129ae02ffe7dee91a5d403e5cb4 WatchSource:0}: Error finding container 8afec5a8c09e9fad1d9971bf31476ed0119d5129ae02ffe7dee91a5d403e5cb4: Status 404 returned error can't find the container with id 8afec5a8c09e9fad1d9971bf31476ed0119d5129ae02ffe7dee91a5d403e5cb4
Jul 17 10:15:19.148059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:19.148035    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"
Jul 17 10:15:20.018429 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.018359    2556 generic.go:356] "Generic (PLEG): container finished" podID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerID="04ef8ec0a434fda232202dfa67d90a6f2601152d3303270ca0c4de8871f216f1" exitCode=6
Jul 17 10:15:20.018429 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.018421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" event={"ID":"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae","Type":"ContainerDied","Data":"04ef8ec0a434fda232202dfa67d90a6f2601152d3303270ca0c4de8871f216f1"}
Jul 17 10:15:20.018855 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.018527    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" event={"ID":"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae","Type":"ContainerStarted","Data":"8afec5a8c09e9fad1d9971bf31476ed0119d5129ae02ffe7dee91a5d403e5cb4"}
Jul 17 10:15:20.018855 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.018770    2556 scope.go:117] "RemoveContainer" containerID="04ef8ec0a434fda232202dfa67d90a6f2601152d3303270ca0c4de8871f216f1"
Jul 17 10:15:20.020277 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.020243    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5" event={"ID":"5730ca08-70a5-48e7-99ff-17cd7acb17ce","Type":"ContainerDied","Data":"c1dad0b57366553b37bee4517be86144a051f5664678c36ef12098daddf43d42"}
Jul 17 10:15:20.020400 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.020290    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1dad0b57366553b37bee4517be86144a051f5664678c36ef12098daddf43d42"
Jul 17 10:15:20.020400 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.020252    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"
Jul 17 10:15:20.061030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.060994    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"]
Jul 17 10:15:20.064606 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.064569    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-1baf67-29738055-kj2f5"]
Jul 17 10:15:20.495524 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.495487    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" path="/var/lib/kubelet/pods/5730ca08-70a5-48e7-99ff-17cd7acb17ce/volumes"
Jul 17 10:15:20.718455 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.718412    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:20.819044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:20.819005    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:21.026966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:21.026925    2556 generic.go:356] "Generic (PLEG): container finished" podID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerID="9998bf73304311f0895239d5b6e8b867d2162bae933aaabf1260d34000b609c0" exitCode=6
Jul 17 10:15:21.027137 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:21.026969    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" event={"ID":"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae","Type":"ContainerDied","Data":"9998bf73304311f0895239d5b6e8b867d2162bae933aaabf1260d34000b609c0"}
Jul 17 10:15:21.027137 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:21.027031    2556 scope.go:117] "RemoveContainer" containerID="04ef8ec0a434fda232202dfa67d90a6f2601152d3303270ca0c4de8871f216f1"
Jul 17 10:15:21.027233 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:21.027185    2556 scope.go:117] "RemoveContainer" containerID="9998bf73304311f0895239d5b6e8b867d2162bae933aaabf1260d34000b609c0"
Jul 17 10:15:21.027412 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:21.027394    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-6spln_odh-ai-gateway-infra(a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae"
Jul 17 10:15:22.033048 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:22.033019    2556 scope.go:117] "RemoveContainer" containerID="9998bf73304311f0895239d5b6e8b867d2162bae933aaabf1260d34000b609c0"
Jul 17 10:15:22.033421 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:22.033183    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-6spln_odh-ai-gateway-infra(a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae"
Jul 17 10:15:22.426415 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:22.426371    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:22.532337 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:22.528875    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:22.714086 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:22.713996    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:22.717231 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:22.717206    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"]
Jul 17 10:15:22.720010 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:22.719987    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-d4c7ee-29738055-7l7l2"]
Jul 17 10:15:24.494687 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:24.494655    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="937e54b9-db3e-4c14-b98f-e1d650c5e3cf" path="/var/lib/kubelet/pods/937e54b9-db3e-4c14-b98f-e1d650c5e3cf/volumes"
Jul 17 10:15:25.080710 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.080674    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:25.877266 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877230    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"]
Jul 17 10:15:25.877623 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877543    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877623 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877555    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877623 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877562    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877623 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877568    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877623 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877577    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877623 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877583    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877814 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877633    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.877814 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.877641    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:25.880252 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.880233    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:25.887206 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:25.887180    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"]
Jul 17 10:15:26.001779 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.001739    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95298408-df3c-482e-bd71-07d69089c42e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-26k82\" (UID: \"95298408-df3c-482e-bd71-07d69089c42e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:26.102179 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.102142    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95298408-df3c-482e-bd71-07d69089c42e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-26k82\" (UID: \"95298408-df3c-482e-bd71-07d69089c42e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:26.102782 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.102761    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95298408-df3c-482e-bd71-07d69089c42e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-26k82\" (UID: \"95298408-df3c-482e-bd71-07d69089c42e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:26.191540 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.191420    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:26.330296 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.330271    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"]
Jul 17 10:15:26.332806 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:26.332779    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95298408_df3c_482e_bd71_07d69089c42e.slice/crio-5194a71a16446cf1ae31c04cf20a53e24803a7f503599e8d7f1231e67ef0d5f1 WatchSource:0}: Error finding container 5194a71a16446cf1ae31c04cf20a53e24803a7f503599e8d7f1231e67ef0d5f1: Status 404 returned error can't find the container with id 5194a71a16446cf1ae31c04cf20a53e24803a7f503599e8d7f1231e67ef0d5f1
Jul 17 10:15:26.458155 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.458057    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:15:26.908163 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:26.908124    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:27.057006 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:27.056972    2556 generic.go:356] "Generic (PLEG): container finished" podID="95298408-df3c-482e-bd71-07d69089c42e" containerID="c2ce4995ea27e44022f88dbe5ba96a0953d1c17144388fb4ef8a46d5d42fc827" exitCode=6
Jul 17 10:15:27.057202 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:27.057056    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" event={"ID":"95298408-df3c-482e-bd71-07d69089c42e","Type":"ContainerDied","Data":"c2ce4995ea27e44022f88dbe5ba96a0953d1c17144388fb4ef8a46d5d42fc827"}
Jul 17 10:15:27.057202 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:27.057118    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" event={"ID":"95298408-df3c-482e-bd71-07d69089c42e","Type":"ContainerStarted","Data":"5194a71a16446cf1ae31c04cf20a53e24803a7f503599e8d7f1231e67ef0d5f1"}
Jul 17 10:15:27.057281 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:27.057204    2556 scope.go:117] "RemoveContainer" containerID="c2ce4995ea27e44022f88dbe5ba96a0953d1c17144388fb4ef8a46d5d42fc827"
Jul 17 10:15:28.063120 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:28.063020    2556 generic.go:356] "Generic (PLEG): container finished" podID="95298408-df3c-482e-bd71-07d69089c42e" containerID="41fb6ea638069fef12175ff29d5c91da202a09352db25d250c3bbd27af867082" exitCode=6
Jul 17 10:15:28.063570 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:28.063117    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" event={"ID":"95298408-df3c-482e-bd71-07d69089c42e","Type":"ContainerDied","Data":"41fb6ea638069fef12175ff29d5c91da202a09352db25d250c3bbd27af867082"}
Jul 17 10:15:28.063570 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:28.063196    2556 scope.go:117] "RemoveContainer" containerID="c2ce4995ea27e44022f88dbe5ba96a0953d1c17144388fb4ef8a46d5d42fc827"
Jul 17 10:15:28.063570 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:28.063426    2556 scope.go:117] "RemoveContainer" containerID="41fb6ea638069fef12175ff29d5c91da202a09352db25d250c3bbd27af867082"
Jul 17 10:15:28.063709 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:28.063664    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-26k82_odh-ai-gateway-infra(95298408-df3c-482e-bd71-07d69089c42e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" podUID="95298408-df3c-482e-bd71-07d69089c42e"
Jul 17 10:15:29.070401 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:29.070366    2556 scope.go:117] "RemoveContainer" containerID="41fb6ea638069fef12175ff29d5c91da202a09352db25d250c3bbd27af867082"
Jul 17 10:15:29.070868 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:29.070556    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-26k82_odh-ai-gateway-infra(95298408-df3c-482e-bd71-07d69089c42e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" podUID="95298408-df3c-482e-bd71-07d69089c42e"
Jul 17 10:15:32.773746 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:32.773711    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:32.874734 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:32.874690    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:33.490198 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:33.490165    2556 scope.go:117] "RemoveContainer" containerID="9998bf73304311f0895239d5b6e8b867d2162bae933aaabf1260d34000b609c0"
Jul 17 10:15:34.094969 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:34.094878    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" event={"ID":"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae","Type":"ContainerStarted","Data":"ddda8e7875c309dde2c2b007d633c8c4310c248f1bde52b778efa377c7cdd5da"}
Jul 17 10:15:34.112619 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:34.112563    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" podStartSLOduration=16.112545064 podStartE2EDuration="16.112545064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:34.111514314 +0000 UTC m=+2918.121638239" watchObservedRunningTime="2026-07-17 10:15:34.112545064 +0000 UTC m=+2918.122668988"
Jul 17 10:15:35.101048 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:35.101009    2556 generic.go:356] "Generic (PLEG): container finished" podID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerID="ddda8e7875c309dde2c2b007d633c8c4310c248f1bde52b778efa377c7cdd5da" exitCode=6
Jul 17 10:15:35.101479 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:35.101087    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" event={"ID":"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae","Type":"ContainerDied","Data":"ddda8e7875c309dde2c2b007d633c8c4310c248f1bde52b778efa377c7cdd5da"}
Jul 17 10:15:35.101479 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:35.101148    2556 scope.go:117] "RemoveContainer" containerID="9998bf73304311f0895239d5b6e8b867d2162bae933aaabf1260d34000b609c0"
Jul 17 10:15:35.101479 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:35.101437    2556 scope.go:117] "RemoveContainer" containerID="ddda8e7875c309dde2c2b007d633c8c4310c248f1bde52b778efa377c7cdd5da"
Jul 17 10:15:35.101682 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:35.101659    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-6spln_odh-ai-gateway-infra(a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae"
Jul 17 10:15:35.127046 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:35.127011    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"]
Jul 17 10:15:36.247254 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.247222    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:36.388741 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.388709    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae-maas-api-service-ca\") pod \"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae\" (UID: \"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae\") "
Jul 17 10:15:36.389063 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.389042    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" (UID: "a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:36.467171 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467129    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"]
Jul 17 10:15:36.467483 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467471    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467544 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467485    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467544 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467495    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467544 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467500    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467544 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467507    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467544 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467512    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467699 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467565    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467699 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467572    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:36.467699 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.467580    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5730ca08-70a5-48e7-99ff-17cd7acb17ce" containerName="cleanup"
Jul 17 10:15:36.470363 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.470342    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:36.478258 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.478050    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"]
Jul 17 10:15:36.489586 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.489550    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:36.590354 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.590313    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf297275-4d54-4ec9-9513-f8f2732386e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb\" (UID: \"bf297275-4d54-4ec9-9513-f8f2732386e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:36.691678 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.691592    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf297275-4d54-4ec9-9513-f8f2732386e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb\" (UID: \"bf297275-4d54-4ec9-9513-f8f2732386e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:36.692246 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.692227    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf297275-4d54-4ec9-9513-f8f2732386e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb\" (UID: \"bf297275-4d54-4ec9-9513-f8f2732386e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:36.781721 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.781695    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:36.919043 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:36.919016    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"]
Jul 17 10:15:36.921770 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:36.921738    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf297275_4d54_4ec9_9513_f8f2732386e8.slice/crio-2f11f5359c97501a7d8459284e1830044124e7c91fca200c2ead1c42d1917b1c WatchSource:0}: Error finding container 2f11f5359c97501a7d8459284e1830044124e7c91fca200c2ead1c42d1917b1c: Status 404 returned error can't find the container with id 2f11f5359c97501a7d8459284e1830044124e7c91fca200c2ead1c42d1917b1c
Jul 17 10:15:37.113500 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.113468    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"
Jul 17 10:15:37.113682 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.113467    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln" event={"ID":"a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae","Type":"ContainerDied","Data":"8afec5a8c09e9fad1d9971bf31476ed0119d5129ae02ffe7dee91a5d403e5cb4"}
Jul 17 10:15:37.113682 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.113614    2556 scope.go:117] "RemoveContainer" containerID="ddda8e7875c309dde2c2b007d633c8c4310c248f1bde52b778efa377c7cdd5da"
Jul 17 10:15:37.115027 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.114986    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" event={"ID":"bf297275-4d54-4ec9-9513-f8f2732386e8","Type":"ContainerStarted","Data":"2f11f5359c97501a7d8459284e1830044124e7c91fca200c2ead1c42d1917b1c"}
Jul 17 10:15:37.130242 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.130215    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"]
Jul 17 10:15:37.137057 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.137029    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-6spln"]
Jul 17 10:15:37.495938 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.495900    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"]
Jul 17 10:15:37.496349 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.496278    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" containerName="revoke-keys"
Jul 17 10:15:37.499259 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.499240    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:37.506473 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.506444    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"]
Jul 17 10:15:37.598187 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.598153    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh\" (UID: \"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:37.610218 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.610178    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:37.699456 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.699393    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh\" (UID: \"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:37.700139 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.700114    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh\" (UID: \"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:37.812183 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.812067    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:37.839633 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.839592    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:37.954099 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.954072    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"]
Jul 17 10:15:37.956868 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:37.956834    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad2e80a_0e3f_4ae7_9a6d_db2416c78a0d.slice/crio-33e6286d78d122d0eb288da71432d809e9d41c08b179faa4b8c5d792b2f629b2 WatchSource:0}: Error finding container 33e6286d78d122d0eb288da71432d809e9d41c08b179faa4b8c5d792b2f629b2: Status 404 returned error can't find the container with id 33e6286d78d122d0eb288da71432d809e9d41c08b179faa4b8c5d792b2f629b2
Jul 17 10:15:37.957384 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.957359    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"]
Jul 17 10:15:37.962385 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.962362    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:37.972345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:37.972130    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"]
Jul 17 10:15:38.102841 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.102716    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7252eb94-3f1c-4373-ba34-645d4ce677fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1baf67-2hx92\" (UID: \"7252eb94-3f1c-4373-ba34-645d4ce677fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:38.120571 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.120535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" event={"ID":"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d","Type":"ContainerStarted","Data":"33e6286d78d122d0eb288da71432d809e9d41c08b179faa4b8c5d792b2f629b2"}
Jul 17 10:15:38.122756 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.122720    2556 generic.go:356] "Generic (PLEG): container finished" podID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerID="e37ced7ea19fc58b083c68c9dc57c5fa8a72ca2332d6a296c0a8711c41cd4f08" exitCode=6
Jul 17 10:15:38.122924 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.122775    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" event={"ID":"bf297275-4d54-4ec9-9513-f8f2732386e8","Type":"ContainerDied","Data":"e37ced7ea19fc58b083c68c9dc57c5fa8a72ca2332d6a296c0a8711c41cd4f08"}
Jul 17 10:15:38.123068 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.123052    2556 scope.go:117] "RemoveContainer" containerID="e37ced7ea19fc58b083c68c9dc57c5fa8a72ca2332d6a296c0a8711c41cd4f08"
Jul 17 10:15:38.203692 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.203651    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7252eb94-3f1c-4373-ba34-645d4ce677fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1baf67-2hx92\" (UID: \"7252eb94-3f1c-4373-ba34-645d4ce677fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:38.204327 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.204299    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7252eb94-3f1c-4373-ba34-645d4ce677fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1baf67-2hx92\" (UID: \"7252eb94-3f1c-4373-ba34-645d4ce677fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:38.276402 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.276362    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:38.429949 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.429859    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"]
Jul 17 10:15:38.432685 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:15:38.432640    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7252eb94_3f1c_4373_ba34_645d4ce677fd.slice/crio-2812a110da8de1f948b0f3ba1abe76fd0cacc604525cad03705d7c38f36cae2b WatchSource:0}: Error finding container 2812a110da8de1f948b0f3ba1abe76fd0cacc604525cad03705d7c38f36cae2b: Status 404 returned error can't find the container with id 2812a110da8de1f948b0f3ba1abe76fd0cacc604525cad03705d7c38f36cae2b
Jul 17 10:15:38.496312 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.496247    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae" path="/var/lib/kubelet/pods/a43f49b6-0b87-4d17-bb0c-aa67eec8a3ae/volumes"
Jul 17 10:15:38.654003 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.653963    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:38.751385 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:38.751347    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:39.129589 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.129551    2556 generic.go:356] "Generic (PLEG): container finished" podID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerID="ce4d595b59a9f87581a9b165caac5ed67a59c21929e38e36febe45d25e966fc9" exitCode=6
Jul 17 10:15:39.129775 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.129639    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" event={"ID":"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d","Type":"ContainerDied","Data":"ce4d595b59a9f87581a9b165caac5ed67a59c21929e38e36febe45d25e966fc9"}
Jul 17 10:15:39.129887 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.129871    2556 scope.go:117] "RemoveContainer" containerID="ce4d595b59a9f87581a9b165caac5ed67a59c21929e38e36febe45d25e966fc9"
Jul 17 10:15:39.131248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.131224    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92" event={"ID":"7252eb94-3f1c-4373-ba34-645d4ce677fd","Type":"ContainerStarted","Data":"94389480342bcfd5ebc7529d5374cc4efde5a9b02bf417702b09b6d3152df0ba"}
Jul 17 10:15:39.131376 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.131274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92" event={"ID":"7252eb94-3f1c-4373-ba34-645d4ce677fd","Type":"ContainerStarted","Data":"2812a110da8de1f948b0f3ba1abe76fd0cacc604525cad03705d7c38f36cae2b"}
Jul 17 10:15:39.133020 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.132996    2556 generic.go:356] "Generic (PLEG): container finished" podID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerID="de1ecd2ff61fcaf51b7cd3d112f9aff7524bf52d43343dde26946938672ad9e0" exitCode=6
Jul 17 10:15:39.133196 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.133034    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" event={"ID":"bf297275-4d54-4ec9-9513-f8f2732386e8","Type":"ContainerDied","Data":"de1ecd2ff61fcaf51b7cd3d112f9aff7524bf52d43343dde26946938672ad9e0"}
Jul 17 10:15:39.133196 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.133078    2556 scope.go:117] "RemoveContainer" containerID="e37ced7ea19fc58b083c68c9dc57c5fa8a72ca2332d6a296c0a8711c41cd4f08"
Jul 17 10:15:39.133343 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.133271    2556 scope.go:117] "RemoveContainer" containerID="de1ecd2ff61fcaf51b7cd3d112f9aff7524bf52d43343dde26946938672ad9e0"
Jul 17 10:15:39.133546 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:39.133466    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb_odh-ai-gateway-infra(bf297275-4d54-4ec9-9513-f8f2732386e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8"
Jul 17 10:15:39.191361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:39.191285    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92" podStartSLOduration=2.191265058 podStartE2EDuration="2.191265058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:39.189729059 +0000 UTC m=+2923.199852984" watchObservedRunningTime="2026-07-17 10:15:39.191265058 +0000 UTC m=+2923.201388981"
Jul 17 10:15:40.139457 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.139418    2556 generic.go:356] "Generic (PLEG): container finished" podID="7252eb94-3f1c-4373-ba34-645d4ce677fd" containerID="94389480342bcfd5ebc7529d5374cc4efde5a9b02bf417702b09b6d3152df0ba" exitCode=0
Jul 17 10:15:40.139884 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.139515    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92" event={"ID":"7252eb94-3f1c-4373-ba34-645d4ce677fd","Type":"ContainerDied","Data":"94389480342bcfd5ebc7529d5374cc4efde5a9b02bf417702b09b6d3152df0ba"}
Jul 17 10:15:40.141443 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.141422    2556 scope.go:117] "RemoveContainer" containerID="de1ecd2ff61fcaf51b7cd3d112f9aff7524bf52d43343dde26946938672ad9e0"
Jul 17 10:15:40.141688 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:40.141666    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb_odh-ai-gateway-infra(bf297275-4d54-4ec9-9513-f8f2732386e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8"
Jul 17 10:15:40.143099 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.143035    2556 generic.go:356] "Generic (PLEG): container finished" podID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerID="cd0b97d2df2b78d2c8c9a41997381a4ac87e3b94f06b712520ac6b54bcbcdf15" exitCode=6
Jul 17 10:15:40.143099 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.143082    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" event={"ID":"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d","Type":"ContainerDied","Data":"cd0b97d2df2b78d2c8c9a41997381a4ac87e3b94f06b712520ac6b54bcbcdf15"}
Jul 17 10:15:40.143278 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.143129    2556 scope.go:117] "RemoveContainer" containerID="ce4d595b59a9f87581a9b165caac5ed67a59c21929e38e36febe45d25e966fc9"
Jul 17 10:15:40.143359 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.143320    2556 scope.go:117] "RemoveContainer" containerID="cd0b97d2df2b78d2c8c9a41997381a4ac87e3b94f06b712520ac6b54bcbcdf15"
Jul 17 10:15:40.143534 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:40.143513    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh_odh-ai-gateway-infra(0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d"
Jul 17 10:15:40.490499 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:40.490406    2556 scope.go:117] "RemoveContainer" containerID="41fb6ea638069fef12175ff29d5c91da202a09352db25d250c3bbd27af867082"
Jul 17 10:15:41.153105 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.153072    2556 scope.go:117] "RemoveContainer" containerID="cd0b97d2df2b78d2c8c9a41997381a4ac87e3b94f06b712520ac6b54bcbcdf15"
Jul 17 10:15:41.153578 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:41.153334    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh_odh-ai-gateway-infra(0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d"
Jul 17 10:15:41.154610 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.154580    2556 generic.go:356] "Generic (PLEG): container finished" podID="95298408-df3c-482e-bd71-07d69089c42e" containerID="c1ae655a9aba7c85d9962d33d8c00bcb83fdde86c7aa9a20d1e739eb333d6331" exitCode=6
Jul 17 10:15:41.154741 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.154655    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" event={"ID":"95298408-df3c-482e-bd71-07d69089c42e","Type":"ContainerDied","Data":"c1ae655a9aba7c85d9962d33d8c00bcb83fdde86c7aa9a20d1e739eb333d6331"}
Jul 17 10:15:41.154741 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.154723    2556 scope.go:117] "RemoveContainer" containerID="41fb6ea638069fef12175ff29d5c91da202a09352db25d250c3bbd27af867082"
Jul 17 10:15:41.155351 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.155079    2556 scope.go:117] "RemoveContainer" containerID="c1ae655a9aba7c85d9962d33d8c00bcb83fdde86c7aa9a20d1e739eb333d6331"
Jul 17 10:15:41.155351 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:41.155252    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-26k82_odh-ai-gateway-infra(95298408-df3c-482e-bd71-07d69089c42e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" podUID="95298408-df3c-482e-bd71-07d69089c42e"
Jul 17 10:15:41.312759 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.312725    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:41.431172 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.431130    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7252eb94-3f1c-4373-ba34-645d4ce677fd-maas-api-service-ca\") pod \"7252eb94-3f1c-4373-ba34-645d4ce677fd\" (UID: \"7252eb94-3f1c-4373-ba34-645d4ce677fd\") "
Jul 17 10:15:41.431576 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.431550    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7252eb94-3f1c-4373-ba34-645d4ce677fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7252eb94-3f1c-4373-ba34-645d4ce677fd" (UID: "7252eb94-3f1c-4373-ba34-645d4ce677fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:41.532642 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:41.532597    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7252eb94-3f1c-4373-ba34-645d4ce677fd-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:42.161412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.161380    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92"
Jul 17 10:15:42.161412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.161402    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1baf67-2hx92" event={"ID":"7252eb94-3f1c-4373-ba34-645d4ce677fd","Type":"ContainerDied","Data":"2812a110da8de1f948b0f3ba1abe76fd0cacc604525cad03705d7c38f36cae2b"}
Jul 17 10:15:42.161855 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.161448    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2812a110da8de1f948b0f3ba1abe76fd0cacc604525cad03705d7c38f36cae2b"
Jul 17 10:15:42.191196 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.191157    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"]
Jul 17 10:15:42.331583 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.331555    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:42.439810 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.439712    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95298408-df3c-482e-bd71-07d69089c42e-maas-api-service-ca\") pod \"95298408-df3c-482e-bd71-07d69089c42e\" (UID: \"95298408-df3c-482e-bd71-07d69089c42e\") "
Jul 17 10:15:42.440118 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.440094    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95298408-df3c-482e-bd71-07d69089c42e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95298408-df3c-482e-bd71-07d69089c42e" (UID: "95298408-df3c-482e-bd71-07d69089c42e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:42.541101 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:42.541059    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95298408-df3c-482e-bd71-07d69089c42e-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:43.172814 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:43.172777    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"
Jul 17 10:15:43.173320 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:43.172784    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82" event={"ID":"95298408-df3c-482e-bd71-07d69089c42e","Type":"ContainerDied","Data":"5194a71a16446cf1ae31c04cf20a53e24803a7f503599e8d7f1231e67ef0d5f1"}
Jul 17 10:15:43.173320 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:43.172932    2556 scope.go:117] "RemoveContainer" containerID="c1ae655a9aba7c85d9962d33d8c00bcb83fdde86c7aa9a20d1e739eb333d6331"
Jul 17 10:15:43.189693 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:43.189661    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"]
Jul 17 10:15:43.195521 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:43.195495    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-26k82"]
Jul 17 10:15:44.494911 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:44.494873    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95298408-df3c-482e-bd71-07d69089c42e" path="/var/lib/kubelet/pods/95298408-df3c-482e-bd71-07d69089c42e/volumes"
Jul 17 10:15:48.453690 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:48.453651    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:51.759206 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:51.759164    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:15:52.489896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:52.489862    2556 scope.go:117] "RemoveContainer" containerID="de1ecd2ff61fcaf51b7cd3d112f9aff7524bf52d43343dde26946938672ad9e0"
Jul 17 10:15:53.057614 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:53.057582    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:53.222624 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:53.222592    2556 generic.go:356] "Generic (PLEG): container finished" podID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerID="1d1994d2dc35e83f919e5207dd36da1334717afbf2d2763e65e08ccae7e10021" exitCode=6
Jul 17 10:15:53.222851 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:53.222664    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" event={"ID":"bf297275-4d54-4ec9-9513-f8f2732386e8","Type":"ContainerDied","Data":"1d1994d2dc35e83f919e5207dd36da1334717afbf2d2763e65e08ccae7e10021"}
Jul 17 10:15:53.222851 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:53.222725    2556 scope.go:117] "RemoveContainer" containerID="de1ecd2ff61fcaf51b7cd3d112f9aff7524bf52d43343dde26946938672ad9e0"
Jul 17 10:15:53.223142 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:53.223126    2556 scope.go:117] "RemoveContainer" containerID="1d1994d2dc35e83f919e5207dd36da1334717afbf2d2763e65e08ccae7e10021"
Jul 17 10:15:53.223339 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:53.223320    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb_odh-ai-gateway-infra(bf297275-4d54-4ec9-9513-f8f2732386e8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8"
Jul 17 10:15:54.248560 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:54.248522    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"]
Jul 17 10:15:54.389724 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:54.389692    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:54.539722 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:54.539625    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf297275-4d54-4ec9-9513-f8f2732386e8-maas-api-service-ca\") pod \"bf297275-4d54-4ec9-9513-f8f2732386e8\" (UID: \"bf297275-4d54-4ec9-9513-f8f2732386e8\") "
Jul 17 10:15:54.540037 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:54.540014    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf297275-4d54-4ec9-9513-f8f2732386e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf297275-4d54-4ec9-9513-f8f2732386e8" (UID: "bf297275-4d54-4ec9-9513-f8f2732386e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:54.641140 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:54.641108    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf297275-4d54-4ec9-9513-f8f2732386e8-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:55.235558 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:55.235516    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"
Jul 17 10:15:55.235944 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:55.235516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb" event={"ID":"bf297275-4d54-4ec9-9513-f8f2732386e8","Type":"ContainerDied","Data":"2f11f5359c97501a7d8459284e1830044124e7c91fca200c2ead1c42d1917b1c"}
Jul 17 10:15:55.235944 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:55.235671    2556 scope.go:117] "RemoveContainer" containerID="1d1994d2dc35e83f919e5207dd36da1334717afbf2d2763e65e08ccae7e10021"
Jul 17 10:15:55.268685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:55.268653    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"]
Jul 17 10:15:55.271419 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:55.271383    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b56xb"]
Jul 17 10:15:55.490290 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:55.490208    2556 scope.go:117] "RemoveContainer" containerID="cd0b97d2df2b78d2c8c9a41997381a4ac87e3b94f06b712520ac6b54bcbcdf15"
Jul 17 10:15:56.243164 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:56.243130    2556 generic.go:356] "Generic (PLEG): container finished" podID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerID="9524877f713fda132c92103edcaa9a224e354b3cb2ba926c11ae572b1be4aaa0" exitCode=6
Jul 17 10:15:56.243340 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:56.243191    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" event={"ID":"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d","Type":"ContainerDied","Data":"9524877f713fda132c92103edcaa9a224e354b3cb2ba926c11ae572b1be4aaa0"}
Jul 17 10:15:56.243340 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:56.243243    2556 scope.go:117] "RemoveContainer" containerID="cd0b97d2df2b78d2c8c9a41997381a4ac87e3b94f06b712520ac6b54bcbcdf15"
Jul 17 10:15:56.243542 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:56.243516    2556 scope.go:117] "RemoveContainer" containerID="9524877f713fda132c92103edcaa9a224e354b3cb2ba926c11ae572b1be4aaa0"
Jul 17 10:15:56.243748 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:15:56.243722    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh_odh-ai-gateway-infra(0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d"
Jul 17 10:15:56.496965 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:56.496935    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" path="/var/lib/kubelet/pods/bf297275-4d54-4ec9-9513-f8f2732386e8/volumes"
Jul 17 10:15:57.091963 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.091925    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:57.189988 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.189952    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:57.268355 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.268317    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"]
Jul 17 10:15:57.410249 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.410222    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:57.569005 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.568950    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d-maas-api-service-ca\") pod \"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d\" (UID: \"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d\") "
Jul 17 10:15:57.569409 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.569274    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" (UID: "0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:57.669532 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:57.669424    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:15:58.255329 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.255295    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"
Jul 17 10:15:58.255480 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.255288    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh" event={"ID":"0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d","Type":"ContainerDied","Data":"33e6286d78d122d0eb288da71432d809e9d41c08b179faa4b8c5d792b2f629b2"}
Jul 17 10:15:58.255480 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.255465    2556 scope.go:117] "RemoveContainer" containerID="9524877f713fda132c92103edcaa9a224e354b3cb2ba926c11ae572b1be4aaa0"
Jul 17 10:15:58.275038 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.275013    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"]
Jul 17 10:15:58.278736 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.278711    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-cgdnh"]
Jul 17 10:15:58.494708 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.494671    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" path="/var/lib/kubelet/pods/0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d/volumes"
Jul 17 10:15:58.890457 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.890416    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:15:58.990458 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:15:58.990422    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:03.753730 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:03.753693    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:06.564626 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:06.564590    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:07.443271 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:07.443223    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:07.634952 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:07.634904    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:07.735338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:07.735239    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:07.835560 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:07.835523    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:08.193838 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.193783    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:08.296970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.296933    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:08.644219 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644181    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"]
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644481    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644492    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644499    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644506    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644517    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644522    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644530    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7252eb94-3f1c-4373-ba34-645d4ce677fd" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644535    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7252eb94-3f1c-4373-ba34-645d4ce677fd" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644543    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644548    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644555    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644560    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644566    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644570    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644576    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644582    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644587    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644591    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644599    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644604    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.644645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644651    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644658    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644666    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644671    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644677    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644683    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644688    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad2e80a-0e3f-4ae7-9a6d-db2416c78a0d" containerName="revoke-keys"
Jul 17 10:16:08.645439 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.644693    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7252eb94-3f1c-4373-ba34-645d4ce677fd" containerName="revoke-keys"
Jul 17 10:16:08.647464 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.647446    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:08.649975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.649952    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:16:08.650065 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.649989    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:16:08.655800 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.655775    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"]
Jul 17 10:16:08.665292 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.665253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa5f3-d6e6-4062-9108-88afaa2c6e14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh\" (UID: \"d38fa5f3-d6e6-4062-9108-88afaa2c6e14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:08.765688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.765638    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa5f3-d6e6-4062-9108-88afaa2c6e14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh\" (UID: \"d38fa5f3-d6e6-4062-9108-88afaa2c6e14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:08.766342 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.766319    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa5f3-d6e6-4062-9108-88afaa2c6e14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh\" (UID: \"d38fa5f3-d6e6-4062-9108-88afaa2c6e14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:08.959438 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:08.959331    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:09.101914 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:09.101885    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"]
Jul 17 10:16:09.105046 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:16:09.104996    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38fa5f3_d6e6_4062_9108_88afaa2c6e14.slice/crio-8dc94e6898b9abec63c2f0227f7e41bc7e3f769cfd053bc7151b2ab2e1e9dedb WatchSource:0}: Error finding container 8dc94e6898b9abec63c2f0227f7e41bc7e3f769cfd053bc7151b2ab2e1e9dedb: Status 404 returned error can't find the container with id 8dc94e6898b9abec63c2f0227f7e41bc7e3f769cfd053bc7151b2ab2e1e9dedb
Jul 17 10:16:09.305937 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:09.305898    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" event={"ID":"d38fa5f3-d6e6-4062-9108-88afaa2c6e14","Type":"ContainerStarted","Data":"8dc94e6898b9abec63c2f0227f7e41bc7e3f769cfd053bc7151b2ab2e1e9dedb"}
Jul 17 10:16:10.311883 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:10.311805    2556 generic.go:356] "Generic (PLEG): container finished" podID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerID="4a0a0ba90f7986a2323964d5d5a343b67865481a47f6378e47ca71e3e536d1cd" exitCode=6
Jul 17 10:16:10.312382 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:10.311900    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" event={"ID":"d38fa5f3-d6e6-4062-9108-88afaa2c6e14","Type":"ContainerDied","Data":"4a0a0ba90f7986a2323964d5d5a343b67865481a47f6378e47ca71e3e536d1cd"}
Jul 17 10:16:10.312382 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:10.312202    2556 scope.go:117] "RemoveContainer" containerID="4a0a0ba90f7986a2323964d5d5a343b67865481a47f6378e47ca71e3e536d1cd"
Jul 17 10:16:11.318116 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:11.318016    2556 generic.go:356] "Generic (PLEG): container finished" podID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerID="7cacf0a03e28b41b8e87be4a624ecbe0f00f5a1a0c29d2d81aca974bfff65bb2" exitCode=6
Jul 17 10:16:11.318116 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:11.318089    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" event={"ID":"d38fa5f3-d6e6-4062-9108-88afaa2c6e14","Type":"ContainerDied","Data":"7cacf0a03e28b41b8e87be4a624ecbe0f00f5a1a0c29d2d81aca974bfff65bb2"}
Jul 17 10:16:11.318650 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:11.318145    2556 scope.go:117] "RemoveContainer" containerID="4a0a0ba90f7986a2323964d5d5a343b67865481a47f6378e47ca71e3e536d1cd"
Jul 17 10:16:11.318650 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:11.318329    2556 scope.go:117] "RemoveContainer" containerID="7cacf0a03e28b41b8e87be4a624ecbe0f00f5a1a0c29d2d81aca974bfff65bb2"
Jul 17 10:16:11.318650 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:11.318592    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh_odh-ai-gateway-infra(d38fa5f3-d6e6-4062-9108-88afaa2c6e14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14"
Jul 17 10:16:11.977584 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:11.977544    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:12.300688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:12.300590    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:12.324550 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:12.324522    2556 scope.go:117] "RemoveContainer" containerID="7cacf0a03e28b41b8e87be4a624ecbe0f00f5a1a0c29d2d81aca974bfff65bb2"
Jul 17 10:16:12.324982 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:12.324716    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh_odh-ai-gateway-infra(d38fa5f3-d6e6-4062-9108-88afaa2c6e14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14"
Jul 17 10:16:12.639574 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:12.639535    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:12.845597 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:12.845567    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:12.960803 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:12.960710    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:16:15.753689 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.753645    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:15.849163 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.849118    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:15.918170 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.918137    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:15.986274 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.986229    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"]
Jul 17 10:16:15.986686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.986667    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf297275-4d54-4ec9-9513-f8f2732386e8" containerName="revoke-keys"
Jul 17 10:16:15.986782 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.986692    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="95298408-df3c-482e-bd71-07d69089c42e" containerName="revoke-keys"
Jul 17 10:16:15.990958 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.990926    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:15.996636 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:15.996609    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"]
Jul 17 10:16:16.020940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.020857    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5feaba1b-0e01-48ae-b55a-0d869859d840-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-mdl7l\" (UID: \"5feaba1b-0e01-48ae-b55a-0d869859d840\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:16.122189 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.122149    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5feaba1b-0e01-48ae-b55a-0d869859d840-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-mdl7l\" (UID: \"5feaba1b-0e01-48ae-b55a-0d869859d840\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:16.122913 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.122883    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5feaba1b-0e01-48ae-b55a-0d869859d840-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-mdl7l\" (UID: \"5feaba1b-0e01-48ae-b55a-0d869859d840\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:16.232570 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.232535    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:16.303518 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.303418    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:16.442485 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.442456    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"]
Jul 17 10:16:16.445402 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:16:16.445371    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5feaba1b_0e01_48ae_b55a_0d869859d840.slice/crio-d485b611f621ff3ec489d2d6ada6ad1b70b0e5341eb069e093474e2dd5bf6256 WatchSource:0}: Error finding container d485b611f621ff3ec489d2d6ada6ad1b70b0e5341eb069e093474e2dd5bf6256: Status 404 returned error can't find the container with id d485b611f621ff3ec489d2d6ada6ad1b70b0e5341eb069e093474e2dd5bf6256
Jul 17 10:16:16.453064 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.453037    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:16.559481 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:16.559382    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:17.348897 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:17.348762    2556 generic.go:356] "Generic (PLEG): container finished" podID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerID="cfd92aa5bf7a2502d64d4dd0d49b63f86969c02a24dc5b356e550f22dd126a70" exitCode=6
Jul 17 10:16:17.348897 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:17.348855    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" event={"ID":"5feaba1b-0e01-48ae-b55a-0d869859d840","Type":"ContainerDied","Data":"cfd92aa5bf7a2502d64d4dd0d49b63f86969c02a24dc5b356e550f22dd126a70"}
Jul 17 10:16:17.349356 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:17.348906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" event={"ID":"5feaba1b-0e01-48ae-b55a-0d869859d840","Type":"ContainerStarted","Data":"d485b611f621ff3ec489d2d6ada6ad1b70b0e5341eb069e093474e2dd5bf6256"}
Jul 17 10:16:17.349356 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:17.349037    2556 scope.go:117] "RemoveContainer" containerID="cfd92aa5bf7a2502d64d4dd0d49b63f86969c02a24dc5b356e550f22dd126a70"
Jul 17 10:16:18.356477 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:18.356381    2556 generic.go:356] "Generic (PLEG): container finished" podID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerID="c298cfef9fbc288c14b755ead66d40fa76efc343ced123b823b761930205c17c" exitCode=6
Jul 17 10:16:18.356918 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:18.356481    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" event={"ID":"5feaba1b-0e01-48ae-b55a-0d869859d840","Type":"ContainerDied","Data":"c298cfef9fbc288c14b755ead66d40fa76efc343ced123b823b761930205c17c"}
Jul 17 10:16:18.356918 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:18.356547    2556 scope.go:117] "RemoveContainer" containerID="cfd92aa5bf7a2502d64d4dd0d49b63f86969c02a24dc5b356e550f22dd126a70"
Jul 17 10:16:18.356918 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:18.356658    2556 scope.go:117] "RemoveContainer" containerID="c298cfef9fbc288c14b755ead66d40fa76efc343ced123b823b761930205c17c"
Jul 17 10:16:18.356918 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:18.356893    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-mdl7l_odh-ai-gateway-infra(5feaba1b-0e01-48ae-b55a-0d869859d840)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840"
Jul 17 10:16:19.363273 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:19.363236    2556 scope.go:117] "RemoveContainer" containerID="c298cfef9fbc288c14b755ead66d40fa76efc343ced123b823b761930205c17c"
Jul 17 10:16:19.363664 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:19.363430    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-mdl7l_odh-ai-gateway-infra(5feaba1b-0e01-48ae-b55a-0d869859d840)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840"
Jul 17 10:16:20.233706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:20.233669    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:23.489756 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:23.489724    2556 scope.go:117] "RemoveContainer" containerID="7cacf0a03e28b41b8e87be4a624ecbe0f00f5a1a0c29d2d81aca974bfff65bb2"
Jul 17 10:16:24.388239 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:24.388207    2556 generic.go:356] "Generic (PLEG): container finished" podID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerID="0a0ace5b6cee39dd1ec6f77f0a6a749cb6d5677bc1a8b117aab8e63a56033eeb" exitCode=6
Jul 17 10:16:24.388239 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:24.388246    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" event={"ID":"d38fa5f3-d6e6-4062-9108-88afaa2c6e14","Type":"ContainerDied","Data":"0a0ace5b6cee39dd1ec6f77f0a6a749cb6d5677bc1a8b117aab8e63a56033eeb"}
Jul 17 10:16:24.388470 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:24.388304    2556 scope.go:117] "RemoveContainer" containerID="7cacf0a03e28b41b8e87be4a624ecbe0f00f5a1a0c29d2d81aca974bfff65bb2"
Jul 17 10:16:24.388641 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:24.388616    2556 scope.go:117] "RemoveContainer" containerID="0a0ace5b6cee39dd1ec6f77f0a6a749cb6d5677bc1a8b117aab8e63a56033eeb"
Jul 17 10:16:24.388895 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:24.388874    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh_odh-ai-gateway-infra(d38fa5f3-d6e6-4062-9108-88afaa2c6e14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14"
Jul 17 10:16:25.414332 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:25.414296    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"]
Jul 17 10:16:25.544773 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:25.544741    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:25.610500 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:25.610460    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa5f3-d6e6-4062-9108-88afaa2c6e14-maas-api-service-ca\") pod \"d38fa5f3-d6e6-4062-9108-88afaa2c6e14\" (UID: \"d38fa5f3-d6e6-4062-9108-88afaa2c6e14\") "
Jul 17 10:16:25.610872 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:25.610849    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38fa5f3-d6e6-4062-9108-88afaa2c6e14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d38fa5f3-d6e6-4062-9108-88afaa2c6e14" (UID: "d38fa5f3-d6e6-4062-9108-88afaa2c6e14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:25.711890 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:25.711785    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d38fa5f3-d6e6-4062-9108-88afaa2c6e14-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:16:25.869123 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:25.869088    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:26.400735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.400693    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh" event={"ID":"d38fa5f3-d6e6-4062-9108-88afaa2c6e14","Type":"ContainerDied","Data":"8dc94e6898b9abec63c2f0227f7e41bc7e3f769cfd053bc7151b2ab2e1e9dedb"}
Jul 17 10:16:26.400735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.400732    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"
Jul 17 10:16:26.401048 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.400807    2556 scope.go:117] "RemoveContainer" containerID="0a0ace5b6cee39dd1ec6f77f0a6a749cb6d5677bc1a8b117aab8e63a56033eeb"
Jul 17 10:16:26.425560 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.425520    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"]
Jul 17 10:16:26.431254 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.431224    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xxnrh"]
Jul 17 10:16:26.494895 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.494861    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" path="/var/lib/kubelet/pods/d38fa5f3-d6e6-4062-9108-88afaa2c6e14/volumes"
Jul 17 10:16:26.511624 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.511589    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"]
Jul 17 10:16:26.511909 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.511896    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:26.511966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.511912    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:26.511966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.511925    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:26.511966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.511933    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:26.512059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.512007    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:26.512059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.512017    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:26.516277 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.516255    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:26.522059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.522025    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"]
Jul 17 10:16:26.619600 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.619554    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1200110d-6d98-4e2f-89e8-dcf99e67da38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl\" (UID: \"1200110d-6d98-4e2f-89e8-dcf99e67da38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:26.720363 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.720281    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1200110d-6d98-4e2f-89e8-dcf99e67da38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl\" (UID: \"1200110d-6d98-4e2f-89e8-dcf99e67da38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:26.720987 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.720961    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1200110d-6d98-4e2f-89e8-dcf99e67da38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl\" (UID: \"1200110d-6d98-4e2f-89e8-dcf99e67da38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:26.828436 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.828391    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:26.972484 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:26.972324    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"]
Jul 17 10:16:26.975226 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:16:26.975195    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1200110d_6d98_4e2f_89e8_dcf99e67da38.slice/crio-aa490de3d4879a65830b55f2ab2868e253cd2fb773bac02301db35995e847cac WatchSource:0}: Error finding container aa490de3d4879a65830b55f2ab2868e253cd2fb773bac02301db35995e847cac: Status 404 returned error can't find the container with id aa490de3d4879a65830b55f2ab2868e253cd2fb773bac02301db35995e847cac
Jul 17 10:16:27.407133 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:27.407093    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" event={"ID":"1200110d-6d98-4e2f-89e8-dcf99e67da38","Type":"ContainerStarted","Data":"91c2cbe1b679325a96f7b30399e9fa2d6c003cc460b3dd40c37db3d1ede5f141"}
Jul 17 10:16:27.407317 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:27.407160    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" event={"ID":"1200110d-6d98-4e2f-89e8-dcf99e67da38","Type":"ContainerStarted","Data":"aa490de3d4879a65830b55f2ab2868e253cd2fb773bac02301db35995e847cac"}
Jul 17 10:16:27.425514 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:27.425455    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" podStartSLOduration=1.425434088 podStartE2EDuration="1.425434088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:16:27.42335171 +0000 UTC m=+2971.433475634" watchObservedRunningTime="2026-07-17 10:16:27.425434088 +0000 UTC m=+2971.435558008"
Jul 17 10:16:27.965930 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:27.965894    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:16:28.414158 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:28.414126    2556 generic.go:356] "Generic (PLEG): container finished" podID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerID="91c2cbe1b679325a96f7b30399e9fa2d6c003cc460b3dd40c37db3d1ede5f141" exitCode=6
Jul 17 10:16:28.414346 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:28.414181    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" event={"ID":"1200110d-6d98-4e2f-89e8-dcf99e67da38","Type":"ContainerDied","Data":"91c2cbe1b679325a96f7b30399e9fa2d6c003cc460b3dd40c37db3d1ede5f141"}
Jul 17 10:16:28.414412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:28.414376    2556 scope.go:117] "RemoveContainer" containerID="91c2cbe1b679325a96f7b30399e9fa2d6c003cc460b3dd40c37db3d1ede5f141"
Jul 17 10:16:29.341081 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:29.340998    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:29.422307 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:29.422267    2556 generic.go:356] "Generic (PLEG): container finished" podID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerID="920263f92b0bfa98bf46728b95f8a6d8b7d6f6424de6429c6ffcdb347a57f76c" exitCode=6
Jul 17 10:16:29.422500 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:29.422361    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" event={"ID":"1200110d-6d98-4e2f-89e8-dcf99e67da38","Type":"ContainerDied","Data":"920263f92b0bfa98bf46728b95f8a6d8b7d6f6424de6429c6ffcdb347a57f76c"}
Jul 17 10:16:29.422500 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:29.422439    2556 scope.go:117] "RemoveContainer" containerID="91c2cbe1b679325a96f7b30399e9fa2d6c003cc460b3dd40c37db3d1ede5f141"
Jul 17 10:16:29.422709 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:29.422684    2556 scope.go:117] "RemoveContainer" containerID="920263f92b0bfa98bf46728b95f8a6d8b7d6f6424de6429c6ffcdb347a57f76c"
Jul 17 10:16:29.422994 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:29.422954    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl_odh-ai-gateway-infra(1200110d-6d98-4e2f-89e8-dcf99e67da38)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38"
Jul 17 10:16:29.443488 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:29.443428    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:30.428958 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:30.428931    2556 scope.go:117] "RemoveContainer" containerID="920263f92b0bfa98bf46728b95f8a6d8b7d6f6424de6429c6ffcdb347a57f76c"
Jul 17 10:16:30.429350 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:30.429125    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl_odh-ai-gateway-infra(1200110d-6d98-4e2f-89e8-dcf99e67da38)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38"
Jul 17 10:16:32.495160 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:32.495119    2556 scope.go:117] "RemoveContainer" containerID="c298cfef9fbc288c14b755ead66d40fa76efc343ced123b823b761930205c17c"
Jul 17 10:16:33.443325 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:33.443283    2556 generic.go:356] "Generic (PLEG): container finished" podID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerID="fad7585111c1c6093700bc934f39297e3745b311faa1efe4e2845b9377323d77" exitCode=6
Jul 17 10:16:33.443513 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:33.443350    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" event={"ID":"5feaba1b-0e01-48ae-b55a-0d869859d840","Type":"ContainerDied","Data":"fad7585111c1c6093700bc934f39297e3745b311faa1efe4e2845b9377323d77"}
Jul 17 10:16:33.443513 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:33.443411    2556 scope.go:117] "RemoveContainer" containerID="c298cfef9fbc288c14b755ead66d40fa76efc343ced123b823b761930205c17c"
Jul 17 10:16:33.443671 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:33.443656    2556 scope.go:117] "RemoveContainer" containerID="fad7585111c1c6093700bc934f39297e3745b311faa1efe4e2845b9377323d77"
Jul 17 10:16:33.443961 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:33.443937    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-mdl7l_odh-ai-gateway-infra(5feaba1b-0e01-48ae-b55a-0d869859d840)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840"
Jul 17 10:16:34.466470 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:34.466436    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"]
Jul 17 10:16:34.604418 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:34.604388    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:34.693434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:34.693393    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5feaba1b-0e01-48ae-b55a-0d869859d840-maas-api-service-ca\") pod \"5feaba1b-0e01-48ae-b55a-0d869859d840\" (UID: \"5feaba1b-0e01-48ae-b55a-0d869859d840\") "
Jul 17 10:16:34.693769 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:34.693744    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5feaba1b-0e01-48ae-b55a-0d869859d840-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5feaba1b-0e01-48ae-b55a-0d869859d840" (UID: "5feaba1b-0e01-48ae-b55a-0d869859d840"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:34.794933 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:34.794811    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5feaba1b-0e01-48ae-b55a-0d869859d840-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:16:35.456507 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:35.456468    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"
Jul 17 10:16:35.456507 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:35.456486    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l" event={"ID":"5feaba1b-0e01-48ae-b55a-0d869859d840","Type":"ContainerDied","Data":"d485b611f621ff3ec489d2d6ada6ad1b70b0e5341eb069e093474e2dd5bf6256"}
Jul 17 10:16:35.456761 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:35.456567    2556 scope.go:117] "RemoveContainer" containerID="fad7585111c1c6093700bc934f39297e3745b311faa1efe4e2845b9377323d77"
Jul 17 10:16:35.484613 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:35.484573    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"]
Jul 17 10:16:35.487005 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:35.486975    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-mdl7l"]
Jul 17 10:16:36.495478 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:36.495441    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" path="/var/lib/kubelet/pods/5feaba1b-0e01-48ae-b55a-0d869859d840/volumes"
Jul 17 10:16:37.234757 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.234725    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:37.335678 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.335637    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:37.742024 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.741989    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"]
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742277    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742288    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742298    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742304    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742320    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742325    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742375    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742381    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742389    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:37.742434 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.742395    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38fa5f3-d6e6-4062-9108-88afaa2c6e14" containerName="revoke-keys"
Jul 17 10:16:37.746198 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.746179    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:37.752685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.752637    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"]
Jul 17 10:16:37.817348 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.817284    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l\" (UID: \"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:37.917960 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.917925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l\" (UID: \"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:37.918580 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:37.918556    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l\" (UID: \"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:38.056979 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:38.056891    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:38.199405 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:38.199349    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"]
Jul 17 10:16:38.203799 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:16:38.203746    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b45ade_e5d7_4f00_bee2_dc1d461f4b3e.slice/crio-e5711584c6e3cd0025ade5792b2b28904beee5210f533b9066aed915f76b67f6 WatchSource:0}: Error finding container e5711584c6e3cd0025ade5792b2b28904beee5210f533b9066aed915f76b67f6: Status 404 returned error can't find the container with id e5711584c6e3cd0025ade5792b2b28904beee5210f533b9066aed915f76b67f6
Jul 17 10:16:38.472606 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:38.472559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" event={"ID":"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e","Type":"ContainerStarted","Data":"e5711584c6e3cd0025ade5792b2b28904beee5210f533b9066aed915f76b67f6"}
Jul 17 10:16:39.479121 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:39.479081    2556 generic.go:356] "Generic (PLEG): container finished" podID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerID="9b14b73e3ca2b49d7cc600af6fb3056f591d41a61b360798fd0e320ae62ef426" exitCode=6
Jul 17 10:16:39.479511 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:39.479173    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" event={"ID":"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e","Type":"ContainerDied","Data":"9b14b73e3ca2b49d7cc600af6fb3056f591d41a61b360798fd0e320ae62ef426"}
Jul 17 10:16:39.479511 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:39.479347    2556 scope.go:117] "RemoveContainer" containerID="9b14b73e3ca2b49d7cc600af6fb3056f591d41a61b360798fd0e320ae62ef426"
Jul 17 10:16:40.485214 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:40.485126    2556 generic.go:356] "Generic (PLEG): container finished" podID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerID="6227acc632bb21416300a08bc3a52e70807df1b5c47bafa458d7ecd5366d8dcf" exitCode=6
Jul 17 10:16:40.485670 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:40.485207    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" event={"ID":"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e","Type":"ContainerDied","Data":"6227acc632bb21416300a08bc3a52e70807df1b5c47bafa458d7ecd5366d8dcf"}
Jul 17 10:16:40.485670 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:40.485269    2556 scope.go:117] "RemoveContainer" containerID="9b14b73e3ca2b49d7cc600af6fb3056f591d41a61b360798fd0e320ae62ef426"
Jul 17 10:16:40.485670 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:40.485498    2556 scope.go:117] "RemoveContainer" containerID="6227acc632bb21416300a08bc3a52e70807df1b5c47bafa458d7ecd5366d8dcf"
Jul 17 10:16:40.485858 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:40.485746    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l_odh-ai-gateway-infra(e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e"
Jul 17 10:16:41.491913 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:41.491879    2556 scope.go:117] "RemoveContainer" containerID="6227acc632bb21416300a08bc3a52e70807df1b5c47bafa458d7ecd5366d8dcf"
Jul 17 10:16:41.492273 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:41.492061    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l_odh-ai-gateway-infra(e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e"
Jul 17 10:16:45.490364 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:45.490335    2556 scope.go:117] "RemoveContainer" containerID="920263f92b0bfa98bf46728b95f8a6d8b7d6f6424de6429c6ffcdb347a57f76c"
Jul 17 10:16:46.516900 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:46.516862    2556 generic.go:356] "Generic (PLEG): container finished" podID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerID="1ebdeb92f8677ef88e5b01cdd086453f03ca364ae73dadc748c7ca461433a583" exitCode=6
Jul 17 10:16:46.517390 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:46.516914    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" event={"ID":"1200110d-6d98-4e2f-89e8-dcf99e67da38","Type":"ContainerDied","Data":"1ebdeb92f8677ef88e5b01cdd086453f03ca364ae73dadc748c7ca461433a583"}
Jul 17 10:16:46.517390 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:46.516963    2556 scope.go:117] "RemoveContainer" containerID="920263f92b0bfa98bf46728b95f8a6d8b7d6f6424de6429c6ffcdb347a57f76c"
Jul 17 10:16:46.517390 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:46.517302    2556 scope.go:117] "RemoveContainer" containerID="1ebdeb92f8677ef88e5b01cdd086453f03ca364ae73dadc748c7ca461433a583"
Jul 17 10:16:46.517619 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:46.517554    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl_odh-ai-gateway-infra(1200110d-6d98-4e2f-89e8-dcf99e67da38)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38"
Jul 17 10:16:47.541245 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:47.541199    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"]
Jul 17 10:16:47.678510 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:47.678474    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:47.803772 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:47.803651    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1200110d-6d98-4e2f-89e8-dcf99e67da38-maas-api-service-ca\") pod \"1200110d-6d98-4e2f-89e8-dcf99e67da38\" (UID: \"1200110d-6d98-4e2f-89e8-dcf99e67da38\") "
Jul 17 10:16:47.804085 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:47.804060    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1200110d-6d98-4e2f-89e8-dcf99e67da38-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1200110d-6d98-4e2f-89e8-dcf99e67da38" (UID: "1200110d-6d98-4e2f-89e8-dcf99e67da38"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:47.905236 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:47.905197    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1200110d-6d98-4e2f-89e8-dcf99e67da38-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:16:48.529069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:48.529021    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"
Jul 17 10:16:48.529069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:48.529032    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl" event={"ID":"1200110d-6d98-4e2f-89e8-dcf99e67da38","Type":"ContainerDied","Data":"aa490de3d4879a65830b55f2ab2868e253cd2fb773bac02301db35995e847cac"}
Jul 17 10:16:48.529366 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:48.529133    2556 scope.go:117] "RemoveContainer" containerID="1ebdeb92f8677ef88e5b01cdd086453f03ca364ae73dadc748c7ca461433a583"
Jul 17 10:16:48.545089 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:48.545049    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"]
Jul 17 10:16:48.548647 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:48.548611    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-9slsl"]
Jul 17 10:16:50.494903 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:50.494864    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" path="/var/lib/kubelet/pods/1200110d-6d98-4e2f-89e8-dcf99e67da38/volumes"
Jul 17 10:16:56.491744 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:56.491712    2556 scope.go:117] "RemoveContainer" containerID="6227acc632bb21416300a08bc3a52e70807df1b5c47bafa458d7ecd5366d8dcf"
Jul 17 10:16:57.573023 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:57.572871    2556 generic.go:356] "Generic (PLEG): container finished" podID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerID="24660007b8105635d31443207f2acd84c075539581bdd159e1a615cc1807c7eb" exitCode=6
Jul 17 10:16:57.573023 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:57.572953    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" event={"ID":"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e","Type":"ContainerDied","Data":"24660007b8105635d31443207f2acd84c075539581bdd159e1a615cc1807c7eb"}
Jul 17 10:16:57.573681 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:57.573080    2556 scope.go:117] "RemoveContainer" containerID="6227acc632bb21416300a08bc3a52e70807df1b5c47bafa458d7ecd5366d8dcf"
Jul 17 10:16:57.573681 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:57.573559    2556 scope.go:117] "RemoveContainer" containerID="24660007b8105635d31443207f2acd84c075539581bdd159e1a615cc1807c7eb"
Jul 17 10:16:57.573897 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:16:57.573868    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l_odh-ai-gateway-infra(e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e"
Jul 17 10:16:58.600840 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.600792    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"]
Jul 17 10:16:58.686745 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.686709    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"]
Jul 17 10:16:58.687035 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687021    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:58.687084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687036    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:58.687084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687051    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:58.687084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687056    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5feaba1b-0e01-48ae-b55a-0d869859d840" containerName="revoke-keys"
Jul 17 10:16:58.687084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687073    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:58.687084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687078    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:58.687235 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687130    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:58.687235 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.687140    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:58.691474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.691437    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:16:58.698969 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.698913    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"]
Jul 17 10:16:58.741093 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.741067    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:58.798380 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.798339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b529a2-470c-48a3-989e-16d5cdb1ed77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rpv24\" (UID: \"33b529a2-470c-48a3-989e-16d5cdb1ed77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:16:58.899142 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.899108    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e-maas-api-service-ca\") pod \"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e\" (UID: \"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e\") "
Jul 17 10:16:58.899314 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.899252    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b529a2-470c-48a3-989e-16d5cdb1ed77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rpv24\" (UID: \"33b529a2-470c-48a3-989e-16d5cdb1ed77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:16:58.899487 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.899460    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" (UID: "e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:58.899840 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:58.899806    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b529a2-470c-48a3-989e-16d5cdb1ed77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rpv24\" (UID: \"33b529a2-470c-48a3-989e-16d5cdb1ed77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:16:59.000103 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.000063    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:16:59.004902 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.004879    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:16:59.170783 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.170747    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"]
Jul 17 10:16:59.173245 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:16:59.173212    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b529a2_470c_48a3_989e_16d5cdb1ed77.slice/crio-06f8ae0b1ad3c613a3d1698370a0752f71c4ffa0e6ac4e8ebc1685ecdc575ffb WatchSource:0}: Error finding container 06f8ae0b1ad3c613a3d1698370a0752f71c4ffa0e6ac4e8ebc1685ecdc575ffb: Status 404 returned error can't find the container with id 06f8ae0b1ad3c613a3d1698370a0752f71c4ffa0e6ac4e8ebc1685ecdc575ffb
Jul 17 10:16:59.362413 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362363    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"]
Jul 17 10:16:59.362803 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362787    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:16:59.362873 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362807    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:16:59.362873 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362852    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:16:59.362873 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362861    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:16:59.362970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362880    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:59.362970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362886    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:59.362970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362953    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1200110d-6d98-4e2f-89e8-dcf99e67da38" containerName="revoke-keys"
Jul 17 10:16:59.362970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362962    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:16:59.362970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.362970    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:16:59.367178 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.367151    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.370180 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.370153    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ed096b\"/\"kube-root-ca.crt\""
Jul 17 10:16:59.371035 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.371008    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed096b\"/\"auth-test-model-ed096b-kserve-self-signed-certs\""
Jul 17 10:16:59.371101 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.371030    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ed096b\"/\"openshift-service-ca.crt\""
Jul 17 10:16:59.371159 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.371007    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed096b\"/\"default-dockercfg-q8dhv\""
Jul 17 10:16:59.375203 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.375172    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"]
Jul 17 10:16:59.386462 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.386428    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:16:59.504224 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.504198    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tmp-dir\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.504361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.504234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-model-cache\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.504361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.504308    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg6m7\" (UniqueName: \"kubernetes.io/projected/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-kube-api-access-vg6m7\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.504361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.504350    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tls-certs\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.504496 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.504390    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-home\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.504496 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.504439    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-dshm\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.584588 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.584548    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" event={"ID":"33b529a2-470c-48a3-989e-16d5cdb1ed77","Type":"ContainerStarted","Data":"05b3c7149be92b8acdc6ce5e385b01e1b2290fc7e4bb9ed63e54282ebf50be34"}
Jul 17 10:16:59.584792 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.584606    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" event={"ID":"33b529a2-470c-48a3-989e-16d5cdb1ed77","Type":"ContainerStarted","Data":"06f8ae0b1ad3c613a3d1698370a0752f71c4ffa0e6ac4e8ebc1685ecdc575ffb"}
Jul 17 10:16:59.585918 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.585892    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"
Jul 17 10:16:59.586054 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.585894    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l" event={"ID":"e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e","Type":"ContainerDied","Data":"e5711584c6e3cd0025ade5792b2b28904beee5210f533b9066aed915f76b67f6"}
Jul 17 10:16:59.586054 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.586030    2556 scope.go:117] "RemoveContainer" containerID="24660007b8105635d31443207f2acd84c075539581bdd159e1a615cc1807c7eb"
Jul 17 10:16:59.599550 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.599508    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" podStartSLOduration=1.5994908799999998 podStartE2EDuration="1.59949088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:16:59.598997901 +0000 UTC m=+3003.609121828" watchObservedRunningTime="2026-07-17 10:16:59.59949088 +0000 UTC m=+3003.609614868"
Jul 17 10:16:59.605453 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605431    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-dshm\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.605792 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tmp-dir\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.605792 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605484    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-model-cache\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.605792 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605529    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg6m7\" (UniqueName: \"kubernetes.io/projected/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-kube-api-access-vg6m7\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.605792 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605560    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tls-certs\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.605792 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-home\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.606082 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605874    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tmp-dir\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.606082 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605915    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-model-cache\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.606082 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.605973    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-home\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.607873 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.607847    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-dshm\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.608120 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.608103    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tls-certs\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.613166 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.613140    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"]
Jul 17 10:16:59.619855 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.619779    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg6m7\" (UniqueName: \"kubernetes.io/projected/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-kube-api-access-vg6m7\") pod \"auth-test-model-ed096b-kserve-755c8d9c87-p7gdm\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") " pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.621836 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.621802    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-tfg4l"]
Jul 17 10:16:59.680190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.680154    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:16:59.822391 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:16:59.822366    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"]
Jul 17 10:16:59.824881 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:16:59.824851    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded2d9ab6_b17c_417c_ae41_f693c32d71ef.slice/crio-ca2379b888f5eae7c76ac3ea3bccdb4da6b9c28c75faf6fbd176e62db859c889 WatchSource:0}: Error finding container ca2379b888f5eae7c76ac3ea3bccdb4da6b9c28c75faf6fbd176e62db859c889: Status 404 returned error can't find the container with id ca2379b888f5eae7c76ac3ea3bccdb4da6b9c28c75faf6fbd176e62db859c889
Jul 17 10:17:00.495321 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.495276    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" path="/var/lib/kubelet/pods/e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e/volumes"
Jul 17 10:17:00.591834 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.591787    2556 generic.go:356] "Generic (PLEG): container finished" podID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerID="05b3c7149be92b8acdc6ce5e385b01e1b2290fc7e4bb9ed63e54282ebf50be34" exitCode=6
Jul 17 10:17:00.592032 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.591877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" event={"ID":"33b529a2-470c-48a3-989e-16d5cdb1ed77","Type":"ContainerDied","Data":"05b3c7149be92b8acdc6ce5e385b01e1b2290fc7e4bb9ed63e54282ebf50be34"}
Jul 17 10:17:00.592103 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.592055    2556 scope.go:117] "RemoveContainer" containerID="05b3c7149be92b8acdc6ce5e385b01e1b2290fc7e4bb9ed63e54282ebf50be34"
Jul 17 10:17:00.593556 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.593536    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm" event={"ID":"ed2d9ab6-b17c-417c-ae41-f693c32d71ef","Type":"ContainerStarted","Data":"4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4"}
Jul 17 10:17:00.593643 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.593567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm" event={"ID":"ed2d9ab6-b17c-417c-ae41-f693c32d71ef","Type":"ContainerStarted","Data":"ca2379b888f5eae7c76ac3ea3bccdb4da6b9c28c75faf6fbd176e62db859c889"}
Jul 17 10:17:00.624320 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:00.624261    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm" podStartSLOduration=1.624242068 podStartE2EDuration="1.624242068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:16:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:00.62324846 +0000 UTC m=+3004.633372384" watchObservedRunningTime="2026-07-17 10:17:00.624242068 +0000 UTC m=+3004.634365993"
Jul 17 10:17:01.601002 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:01.600912    2556 generic.go:356] "Generic (PLEG): container finished" podID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerID="60b9c8af36b128399e8f1be946fac7ebecd3b73e5127d310741ad37f3a61b901" exitCode=6
Jul 17 10:17:01.601157 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:01.601004    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" event={"ID":"33b529a2-470c-48a3-989e-16d5cdb1ed77","Type":"ContainerDied","Data":"60b9c8af36b128399e8f1be946fac7ebecd3b73e5127d310741ad37f3a61b901"}
Jul 17 10:17:01.601157 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:01.601074    2556 scope.go:117] "RemoveContainer" containerID="05b3c7149be92b8acdc6ce5e385b01e1b2290fc7e4bb9ed63e54282ebf50be34"
Jul 17 10:17:01.601284 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:01.601254    2556 scope.go:117] "RemoveContainer" containerID="60b9c8af36b128399e8f1be946fac7ebecd3b73e5127d310741ad37f3a61b901"
Jul 17 10:17:01.601476 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:17:01.601455    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rpv24_odh-ai-gateway-infra(33b529a2-470c-48a3-989e-16d5cdb1ed77)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77"
Jul 17 10:17:02.609204 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:02.609169    2556 scope.go:117] "RemoveContainer" containerID="60b9c8af36b128399e8f1be946fac7ebecd3b73e5127d310741ad37f3a61b901"
Jul 17 10:17:02.609590 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:17:02.609365    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rpv24_odh-ai-gateway-infra(33b529a2-470c-48a3-989e-16d5cdb1ed77)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77"
Jul 17 10:17:02.857086 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:02.857043    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:17:06.027707 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.027667    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"]
Jul 17 10:17:06.028167 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.027989    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:17:06.028167 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.028000    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:17:06.028167 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.028062    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b45ade-e5d7-4f00-bee2-dc1d461f4b3e" containerName="revoke-keys"
Jul 17 10:17:06.031032 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.031013    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:06.039681 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.039651    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"]
Jul 17 10:17:06.060474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.060426    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/827447fb-dc89-4d07-9334-63310d93535e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-4prsb\" (UID: \"827447fb-dc89-4d07-9334-63310d93535e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:06.160953 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.160904    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/827447fb-dc89-4d07-9334-63310d93535e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-4prsb\" (UID: \"827447fb-dc89-4d07-9334-63310d93535e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:06.161572 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.161550    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/827447fb-dc89-4d07-9334-63310d93535e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-4prsb\" (UID: \"827447fb-dc89-4d07-9334-63310d93535e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:06.342527 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.342431    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:06.483262 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.483049    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"]
Jul 17 10:17:06.485632 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:17:06.485602    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod827447fb_dc89_4d07_9334_63310d93535e.slice/crio-0fdb08cbca168e27f977bbadd5664988f230eef97f6e71cc3b0d7a4a24cbcce8 WatchSource:0}: Error finding container 0fdb08cbca168e27f977bbadd5664988f230eef97f6e71cc3b0d7a4a24cbcce8: Status 404 returned error can't find the container with id 0fdb08cbca168e27f977bbadd5664988f230eef97f6e71cc3b0d7a4a24cbcce8
Jul 17 10:17:06.633939 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:06.633901    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" event={"ID":"827447fb-dc89-4d07-9334-63310d93535e","Type":"ContainerStarted","Data":"0fdb08cbca168e27f977bbadd5664988f230eef97f6e71cc3b0d7a4a24cbcce8"}
Jul 17 10:17:07.640157 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:07.640115    2556 generic.go:356] "Generic (PLEG): container finished" podID="827447fb-dc89-4d07-9334-63310d93535e" containerID="9e8d36c81f45e752ec66797dda340472b558bbe4dc115b5f9c8e6f248489ad42" exitCode=6
Jul 17 10:17:07.640557 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:07.640205    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" event={"ID":"827447fb-dc89-4d07-9334-63310d93535e","Type":"ContainerDied","Data":"9e8d36c81f45e752ec66797dda340472b558bbe4dc115b5f9c8e6f248489ad42"}
Jul 17 10:17:07.640557 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:07.640368    2556 scope.go:117] "RemoveContainer" containerID="9e8d36c81f45e752ec66797dda340472b558bbe4dc115b5f9c8e6f248489ad42"
Jul 17 10:17:08.646379 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:08.646343    2556 generic.go:356] "Generic (PLEG): container finished" podID="827447fb-dc89-4d07-9334-63310d93535e" containerID="c7f8d31efa4a186accdabf8d26aa9ca85f51bf9b3aa2554d147eec358221a58b" exitCode=6
Jul 17 10:17:08.646807 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:08.646393    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" event={"ID":"827447fb-dc89-4d07-9334-63310d93535e","Type":"ContainerDied","Data":"c7f8d31efa4a186accdabf8d26aa9ca85f51bf9b3aa2554d147eec358221a58b"}
Jul 17 10:17:08.646807 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:08.646457    2556 scope.go:117] "RemoveContainer" containerID="9e8d36c81f45e752ec66797dda340472b558bbe4dc115b5f9c8e6f248489ad42"
Jul 17 10:17:08.646807 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:08.646621    2556 scope.go:117] "RemoveContainer" containerID="c7f8d31efa4a186accdabf8d26aa9ca85f51bf9b3aa2554d147eec358221a58b"
Jul 17 10:17:08.646950 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:17:08.646866    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-4prsb_odh-ai-gateway-infra(827447fb-dc89-4d07-9334-63310d93535e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" podUID="827447fb-dc89-4d07-9334-63310d93535e"
Jul 17 10:17:09.654190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:09.654162    2556 scope.go:117] "RemoveContainer" containerID="c7f8d31efa4a186accdabf8d26aa9ca85f51bf9b3aa2554d147eec358221a58b"
Jul 17 10:17:09.654575 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:17:09.654387    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-4prsb_odh-ai-gateway-infra(827447fb-dc89-4d07-9334-63310d93535e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" podUID="827447fb-dc89-4d07-9334-63310d93535e"
Jul 17 10:17:09.680473 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:09.680424    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:17:09.680653 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:09.680499    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:17:09.693605 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:09.693576    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:17:10.669746 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:10.669714    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:17:16.491696 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.491610    2556 scope.go:117] "RemoveContainer" containerID="60b9c8af36b128399e8f1be946fac7ebecd3b73e5127d310741ad37f3a61b901"
Jul 17 10:17:16.569724 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.569686    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"]
Jul 17 10:17:16.574461 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.574431    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.577016 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.576990    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"kube-root-ca.crt\""
Jul 17 10:17:16.577171 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.577043    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"openshift-service-ca.crt\""
Jul 17 10:17:16.577171 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.577096    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"auth-test-model-af80d9-kserve-self-signed-certs\""
Jul 17 10:17:16.577171 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.577045    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"default-dockercfg-s6q2v\""
Jul 17 10:17:16.582416 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.582275    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"]
Jul 17 10:17:16.656406 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.656363    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-dshm\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.656608 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.656435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-model-cache\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.656608 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.656479    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tls-certs\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.656608 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.656530    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmfg8\" (UniqueName: \"kubernetes.io/projected/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-kube-api-access-fmfg8\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.656608 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.656571    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-home\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.656608 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.656593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tmp-dir\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.667120 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.667090    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:17:16.757896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.757761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-model-cache\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.757896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.757844    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tls-certs\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.757896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.757885    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmfg8\" (UniqueName: \"kubernetes.io/projected/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-kube-api-access-fmfg8\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.758193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.757911    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-home\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.758193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.757933    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tmp-dir\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.758193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.757980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-dshm\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.758352 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.758249    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-model-cache\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.758352 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.758301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-home\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.758352 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.758336    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tmp-dir\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.760563 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.760540    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-dshm\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.760694 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.760676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tls-certs\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.766710 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.766675    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmfg8\" (UniqueName: \"kubernetes.io/projected/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-kube-api-access-fmfg8\") pod \"auth-test-model-af80d9-kserve-867c4c45f-csc6j\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") " pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:16.886402 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:16.886366    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:17.031202 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.031161    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"]
Jul 17 10:17:17.033244 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:17:17.033214    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod883d7b5c_1ccb_43ff_886a_cdee8a76ed4e.slice/crio-60f833a6ee6c8eb83c734776198816e067c3c05632fc43179b8029fed6cb56f5 WatchSource:0}: Error finding container 60f833a6ee6c8eb83c734776198816e067c3c05632fc43179b8029fed6cb56f5: Status 404 returned error can't find the container with id 60f833a6ee6c8eb83c734776198816e067c3c05632fc43179b8029fed6cb56f5
Jul 17 10:17:17.692641 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.692609    2556 generic.go:356] "Generic (PLEG): container finished" podID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerID="9ef95314e4333849acb765fc536d0281227fdfcf507fa7f25c357068ec22eb64" exitCode=6
Jul 17 10:17:17.693139 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.692691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" event={"ID":"33b529a2-470c-48a3-989e-16d5cdb1ed77","Type":"ContainerDied","Data":"9ef95314e4333849acb765fc536d0281227fdfcf507fa7f25c357068ec22eb64"}
Jul 17 10:17:17.693139 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.692766    2556 scope.go:117] "RemoveContainer" containerID="60b9c8af36b128399e8f1be946fac7ebecd3b73e5127d310741ad37f3a61b901"
Jul 17 10:17:17.693139 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.693128    2556 scope.go:117] "RemoveContainer" containerID="9ef95314e4333849acb765fc536d0281227fdfcf507fa7f25c357068ec22eb64"
Jul 17 10:17:17.693429 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:17:17.693348    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rpv24_odh-ai-gateway-infra(33b529a2-470c-48a3-989e-16d5cdb1ed77)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77"
Jul 17 10:17:17.694615 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.694586    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j" event={"ID":"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e","Type":"ContainerStarted","Data":"c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2"}
Jul 17 10:17:17.694784 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.694647    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j" event={"ID":"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e","Type":"ContainerStarted","Data":"60f833a6ee6c8eb83c734776198816e067c3c05632fc43179b8029fed6cb56f5"}
Jul 17 10:17:17.726338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:17.726286    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j" podStartSLOduration=1.726266625 podStartE2EDuration="1.726266625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:17.725537219 +0000 UTC m=+3021.735661143" watchObservedRunningTime="2026-07-17 10:17:17.726266625 +0000 UTC m=+3021.736390549"
Jul 17 10:17:18.718428 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:18.718391    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"]
Jul 17 10:17:18.866078 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:18.866038    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:17:18.976140 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:18.976045    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b529a2-470c-48a3-989e-16d5cdb1ed77-maas-api-service-ca\") pod \"33b529a2-470c-48a3-989e-16d5cdb1ed77\" (UID: \"33b529a2-470c-48a3-989e-16d5cdb1ed77\") "
Jul 17 10:17:18.976426 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:18.976400    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33b529a2-470c-48a3-989e-16d5cdb1ed77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33b529a2-470c-48a3-989e-16d5cdb1ed77" (UID: "33b529a2-470c-48a3-989e-16d5cdb1ed77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:17:19.077461 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:19.077424    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33b529a2-470c-48a3-989e-16d5cdb1ed77-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:17:19.712248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:19.712205    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"
Jul 17 10:17:19.712526 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:19.712198    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24" event={"ID":"33b529a2-470c-48a3-989e-16d5cdb1ed77","Type":"ContainerDied","Data":"06f8ae0b1ad3c613a3d1698370a0752f71c4ffa0e6ac4e8ebc1685ecdc575ffb"}
Jul 17 10:17:19.712526 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:19.712444    2556 scope.go:117] "RemoveContainer" containerID="9ef95314e4333849acb765fc536d0281227fdfcf507fa7f25c357068ec22eb64"
Jul 17 10:17:19.736287 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:19.736248    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"]
Jul 17 10:17:19.739864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:19.739815    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpv24"]
Jul 17 10:17:20.263456 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:20.263415    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:17:20.495882 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:20.495846    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" path="/var/lib/kubelet/pods/33b529a2-470c-48a3-989e-16d5cdb1ed77/volumes"
Jul 17 10:17:22.490104 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:22.490066    2556 scope.go:117] "RemoveContainer" containerID="c7f8d31efa4a186accdabf8d26aa9ca85f51bf9b3aa2554d147eec358221a58b"
Jul 17 10:17:23.732859 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:23.732803    2556 generic.go:356] "Generic (PLEG): container finished" podID="827447fb-dc89-4d07-9334-63310d93535e" containerID="b79cf27426250dd76377c42eff5edc64b6d72168a039596bde653d60e7b5f65b" exitCode=6
Jul 17 10:17:23.733329 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:23.732881    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" event={"ID":"827447fb-dc89-4d07-9334-63310d93535e","Type":"ContainerDied","Data":"b79cf27426250dd76377c42eff5edc64b6d72168a039596bde653d60e7b5f65b"}
Jul 17 10:17:23.733329 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:23.732941    2556 scope.go:117] "RemoveContainer" containerID="c7f8d31efa4a186accdabf8d26aa9ca85f51bf9b3aa2554d147eec358221a58b"
Jul 17 10:17:23.733329 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:23.733314    2556 scope.go:117] "RemoveContainer" containerID="b79cf27426250dd76377c42eff5edc64b6d72168a039596bde653d60e7b5f65b"
Jul 17 10:17:23.733592 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:17:23.733567    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-4prsb_odh-ai-gateway-infra(827447fb-dc89-4d07-9334-63310d93535e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" podUID="827447fb-dc89-4d07-9334-63310d93535e"
Jul 17 10:17:24.757438 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:24.757392    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"]
Jul 17 10:17:24.896945 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:24.896913    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:24.929038 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:24.928985    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/827447fb-dc89-4d07-9334-63310d93535e-maas-api-service-ca\") pod \"827447fb-dc89-4d07-9334-63310d93535e\" (UID: \"827447fb-dc89-4d07-9334-63310d93535e\") "
Jul 17 10:17:24.929470 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:24.929437    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/827447fb-dc89-4d07-9334-63310d93535e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "827447fb-dc89-4d07-9334-63310d93535e" (UID: "827447fb-dc89-4d07-9334-63310d93535e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:17:25.030576 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:25.030494    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/827447fb-dc89-4d07-9334-63310d93535e-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:17:25.745487 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:25.745451    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"
Jul 17 10:17:25.745684 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:25.745450    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb" event={"ID":"827447fb-dc89-4d07-9334-63310d93535e","Type":"ContainerDied","Data":"0fdb08cbca168e27f977bbadd5664988f230eef97f6e71cc3b0d7a4a24cbcce8"}
Jul 17 10:17:25.745684 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:25.745601    2556 scope.go:117] "RemoveContainer" containerID="b79cf27426250dd76377c42eff5edc64b6d72168a039596bde653d60e7b5f65b"
Jul 17 10:17:25.766229 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:25.766184    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"]
Jul 17 10:17:25.772354 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:25.772323    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-4prsb"]
Jul 17 10:17:26.495113 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:26.495076    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="827447fb-dc89-4d07-9334-63310d93535e" path="/var/lib/kubelet/pods/827447fb-dc89-4d07-9334-63310d93535e/volumes"
Jul 17 10:17:26.887504 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:26.887468    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:26.887950 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:26.887524    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:26.900417 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:26.900385    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:27.766228 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:27.766199    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:17:28.391179 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:28.391146    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:17:34.566770 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:34.566725    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:17:45.763359 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:45.763320    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:17:45.925394 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:45.925356    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:17:49.762095 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:17:49.762006    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:18:08.957583 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:18:08.957544    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:18:29.168183 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:18:29.168146    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:18:48.665053 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:18:48.665010    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:19:08.464145 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:08.464098    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:19:23.518581 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:23.518538    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"]
Jul 17 10:19:23.569619 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:23.569580    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:19:24.549975 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:24.549935    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"]
Jul 17 10:19:24.569922 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:24.569881    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:19:28.057196 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057160    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"]
Jul 17 10:19:28.057663 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057645    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057715 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057668    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057715 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057689    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057715 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057699    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057715 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057710    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057719    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057729    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057737    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057751    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057761    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057772    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057780    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057882    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057894    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057906    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.057916 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057915    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.058485 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057923    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="827447fb-dc89-4d07-9334-63310d93535e" containerName="revoke-keys"
Jul 17 10:19:28.058485 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.057934    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b529a2-470c-48a3-989e-16d5cdb1ed77" containerName="revoke-keys"
Jul 17 10:19:28.061408 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.061385    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.063993 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.063965    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed096b\"/\"sub-test-model-ed096b-kserve-self-signed-certs\""
Jul 17 10:19:28.069478 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.069444    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"]
Jul 17 10:19:28.091591 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.091555    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:19:28.173465 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.173422    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-model-cache\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.173465 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.173465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tls-certs\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.173706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.173525    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-dshm\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.173706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.173573    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tmp-dir\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.173706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.173593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbz7r\" (UniqueName: \"kubernetes.io/projected/f3ccd25c-a65c-443f-8de4-bad9211a78f5-kube-api-access-fbz7r\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.173706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.173622    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-home\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275012 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.274970    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-home\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275051    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-model-cache\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tls-certs\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275088    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-dshm\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275109    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tmp-dir\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbz7r\" (UniqueName: \"kubernetes.io/projected/f3ccd25c-a65c-443f-8de4-bad9211a78f5-kube-api-access-fbz7r\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275449 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275427    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-home\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275483    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tmp-dir\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.275594 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.275573    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-model-cache\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.277488 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.277470    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-dshm\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.277747 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.277729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tls-certs\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.283643 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.283619    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbz7r\" (UniqueName: \"kubernetes.io/projected/f3ccd25c-a65c-443f-8de4-bad9211a78f5-kube-api-access-fbz7r\") pod \"sub-test-model-ed096b-kserve-6564d45867-pqxnn\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") " pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.373764 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.373711    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:28.515509 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.515481    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"]
Jul 17 10:19:28.518300 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:19:28.518271    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3ccd25c_a65c_443f_8de4_bad9211a78f5.slice/crio-b6c9051a7f267cec79ed382cb13bd8d74c1e7af6d12b8d73318a04df67e54622 WatchSource:0}: Error finding container b6c9051a7f267cec79ed382cb13bd8d74c1e7af6d12b8d73318a04df67e54622: Status 404 returned error can't find the container with id b6c9051a7f267cec79ed382cb13bd8d74c1e7af6d12b8d73318a04df67e54622
Jul 17 10:19:28.568358 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:28.568333    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:19:29.324956 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:29.324917    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn" event={"ID":"f3ccd25c-a65c-443f-8de4-bad9211a78f5","Type":"ContainerStarted","Data":"c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823"}
Jul 17 10:19:29.324956 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:29.324962    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn" event={"ID":"f3ccd25c-a65c-443f-8de4-bad9211a78f5","Type":"ContainerStarted","Data":"b6c9051a7f267cec79ed382cb13bd8d74c1e7af6d12b8d73318a04df67e54622"}
Jul 17 10:19:29.344085 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:29.344033    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn" podStartSLOduration=1.344016544 podStartE2EDuration="1.344016544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:19:29.343268187 +0000 UTC m=+3153.353392107" watchObservedRunningTime="2026-07-17 10:19:29.344016544 +0000 UTC m=+3153.354140467"
Jul 17 10:19:38.374709 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.374671    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:38.375210 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.374866    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:38.387206 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.387176    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:38.540703 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.540641    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm" podUID="ed2d9ab6-b17c-417c-ae41-f693c32d71ef" containerName="main" containerID="cri-o://4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4" gracePeriod=45
Jul 17 10:19:38.804174 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.804147    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:19:38.868589 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868548    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tls-certs\") pod \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") "
Jul 17 10:19:38.868589 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868593    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tmp-dir\") pod \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") "
Jul 17 10:19:38.868886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868612    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-dshm\") pod \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") "
Jul 17 10:19:38.868886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868649    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-home\") pod \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") "
Jul 17 10:19:38.868886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868687    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg6m7\" (UniqueName: \"kubernetes.io/projected/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-kube-api-access-vg6m7\") pod \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") "
Jul 17 10:19:38.868886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868737    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-model-cache\") pod \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\" (UID: \"ed2d9ab6-b17c-417c-ae41-f693c32d71ef\") "
Jul 17 10:19:38.869107 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868890    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed2d9ab6-b17c-417c-ae41-f693c32d71ef" (UID: "ed2d9ab6-b17c-417c-ae41-f693c32d71ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:38.869107 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.868940    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-home" (OuterVolumeSpecName: "home") pod "ed2d9ab6-b17c-417c-ae41-f693c32d71ef" (UID: "ed2d9ab6-b17c-417c-ae41-f693c32d71ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:38.869107 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.869090    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed2d9ab6-b17c-417c-ae41-f693c32d71ef" (UID: "ed2d9ab6-b17c-417c-ae41-f693c32d71ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:38.869256 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.869155    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:38.869256 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.869172    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:38.869256 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.869187    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:38.871211 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.871178    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed2d9ab6-b17c-417c-ae41-f693c32d71ef" (UID: "ed2d9ab6-b17c-417c-ae41-f693c32d71ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:19:38.871211 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.871182    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-dshm" (OuterVolumeSpecName: "dshm") pod "ed2d9ab6-b17c-417c-ae41-f693c32d71ef" (UID: "ed2d9ab6-b17c-417c-ae41-f693c32d71ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:38.871350 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.871230    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-kube-api-access-vg6m7" (OuterVolumeSpecName: "kube-api-access-vg6m7") pod "ed2d9ab6-b17c-417c-ae41-f693c32d71ef" (UID: "ed2d9ab6-b17c-417c-ae41-f693c32d71ef"). InnerVolumeSpecName "kube-api-access-vg6m7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:19:38.970338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.970247    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:38.970338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.970277    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:38.970338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:38.970286    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg6m7\" (UniqueName: \"kubernetes.io/projected/ed2d9ab6-b17c-417c-ae41-f693c32d71ef-kube-api-access-vg6m7\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:39.381110 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.381070    2556 generic.go:356] "Generic (PLEG): container finished" podID="ed2d9ab6-b17c-417c-ae41-f693c32d71ef" containerID="4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4" exitCode=0
Jul 17 10:19:39.385017 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.382162    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"
Jul 17 10:19:39.385017 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.382577    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm" event={"ID":"ed2d9ab6-b17c-417c-ae41-f693c32d71ef","Type":"ContainerDied","Data":"4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4"}
Jul 17 10:19:39.385017 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.382625    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm" event={"ID":"ed2d9ab6-b17c-417c-ae41-f693c32d71ef","Type":"ContainerDied","Data":"ca2379b888f5eae7c76ac3ea3bccdb4da6b9c28c75faf6fbd176e62db859c889"}
Jul 17 10:19:39.385017 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.382646    2556 scope.go:117] "RemoveContainer" containerID="4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4"
Jul 17 10:19:39.405794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.405434    2556 scope.go:117] "RemoveContainer" containerID="4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4"
Jul 17 10:19:39.406360 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:19:39.406292    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4\": container with ID starting with 4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4 not found: ID does not exist" containerID="4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4"
Jul 17 10:19:39.406360 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.406330    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4"} err="failed to get container status \"4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4\": rpc error: code = NotFound desc = could not find container \"4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4\": container with ID starting with 4ee1eca4113895c877acc5a7b12385a9bb879be5d5896c1ce5552396557120f4 not found: ID does not exist"
Jul 17 10:19:39.408286 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.408255    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:19:39.420698 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.420621    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"]
Jul 17 10:19:39.425489 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.425443    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/auth-test-model-ed096b-kserve-755c8d9c87-p7gdm"]
Jul 17 10:19:39.573198 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.573148    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j" podUID="883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" containerName="main" containerID="cri-o://c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2" gracePeriod=45
Jul 17 10:19:39.832846 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.832791    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:19:39.981588 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981495    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tls-certs\") pod \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") "
Jul 17 10:19:39.981588 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981545    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-dshm\") pod \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") "
Jul 17 10:19:39.981588 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981564    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-model-cache\") pod \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") "
Jul 17 10:19:39.981588 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981589    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tmp-dir\") pod \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") "
Jul 17 10:19:39.981943 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981713    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmfg8\" (UniqueName: \"kubernetes.io/projected/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-kube-api-access-fmfg8\") pod \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") "
Jul 17 10:19:39.981943 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981762    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-home\") pod \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\" (UID: \"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e\") "
Jul 17 10:19:39.981943 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981800    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" (UID: "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:39.981943 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981875    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" (UID: "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:39.982141 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.981994    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-home" (OuterVolumeSpecName: "home") pod "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" (UID: "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:39.982141 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.982130    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:39.982225 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.982146    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:39.982225 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.982159    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:39.984036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.984005    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-dshm" (OuterVolumeSpecName: "dshm") pod "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" (UID: "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:19:39.984133 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.984008    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" (UID: "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:19:39.984133 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:39.984100    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-kube-api-access-fmfg8" (OuterVolumeSpecName: "kube-api-access-fmfg8") pod "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" (UID: "883d7b5c-1ccb-43ff-886a-cdee8a76ed4e"). InnerVolumeSpecName "kube-api-access-fmfg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:19:40.082802 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.082760    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmfg8\" (UniqueName: \"kubernetes.io/projected/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-kube-api-access-fmfg8\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:40.082802 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.082793    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:40.082802 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.082804    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:19:40.388894 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.388858    2556 generic.go:356] "Generic (PLEG): container finished" podID="883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" containerID="c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2" exitCode=0
Jul 17 10:19:40.389385 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.388920    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"
Jul 17 10:19:40.389385 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.388942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j" event={"ID":"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e","Type":"ContainerDied","Data":"c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2"}
Jul 17 10:19:40.389385 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.388981    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j" event={"ID":"883d7b5c-1ccb-43ff-886a-cdee8a76ed4e","Type":"ContainerDied","Data":"60f833a6ee6c8eb83c734776198816e067c3c05632fc43179b8029fed6cb56f5"}
Jul 17 10:19:40.389385 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.388998    2556 scope.go:117] "RemoveContainer" containerID="c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2"
Jul 17 10:19:40.398230 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.398209    2556 scope.go:117] "RemoveContainer" containerID="c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2"
Jul 17 10:19:40.398617 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:19:40.398592    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2\": container with ID starting with c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2 not found: ID does not exist" containerID="c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2"
Jul 17 10:19:40.398749 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.398624    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2"} err="failed to get container status \"c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2\": rpc error: code = NotFound desc = could not find container \"c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2\": container with ID starting with c5dedfd9a49a88f2cbaf7cfcf2e302e3ed06e843e38b08fe0db8aec47ee7f0b2 not found: ID does not exist"
Jul 17 10:19:40.413337 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.413296    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"]
Jul 17 10:19:40.417467 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.417436    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/auth-test-model-af80d9-kserve-867c4c45f-csc6j"]
Jul 17 10:19:40.495293 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.495257    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" path="/var/lib/kubelet/pods/883d7b5c-1ccb-43ff-886a-cdee8a76ed4e/volumes"
Jul 17 10:19:40.495720 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:40.495699    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed2d9ab6-b17c-417c-ae41-f693c32d71ef" path="/var/lib/kubelet/pods/ed2d9ab6-b17c-417c-ae41-f693c32d71ef/volumes"
Jul 17 10:19:46.264875 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:46.264788    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:19:56.688766 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.688718    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"]
Jul 17 10:19:56.689282 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.689149    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" containerName="main"
Jul 17 10:19:56.689282 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.689169    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" containerName="main"
Jul 17 10:19:56.689282 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.689192    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed2d9ab6-b17c-417c-ae41-f693c32d71ef" containerName="main"
Jul 17 10:19:56.689282 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.689201    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed2d9ab6-b17c-417c-ae41-f693c32d71ef" containerName="main"
Jul 17 10:19:56.689492 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.689285    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="883d7b5c-1ccb-43ff-886a-cdee8a76ed4e" containerName="main"
Jul 17 10:19:56.689492 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.689300    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed2d9ab6-b17c-417c-ae41-f693c32d71ef" containerName="main"
Jul 17 10:19:56.693724 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.693698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:19:56.696351 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.696322    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:19:56.697141 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.697115    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:19:56.698386 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.698358    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"]
Jul 17 10:19:56.823142 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.823099    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf687553-b4df-4672-81eb-c1cffbf4e66f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm\" (UID: \"cf687553-b4df-4672-81eb-c1cffbf4e66f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:19:56.923923 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.923887    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf687553-b4df-4672-81eb-c1cffbf4e66f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm\" (UID: \"cf687553-b4df-4672-81eb-c1cffbf4e66f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:19:56.924562 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:56.924539    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf687553-b4df-4672-81eb-c1cffbf4e66f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm\" (UID: \"cf687553-b4df-4672-81eb-c1cffbf4e66f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:19:57.006011 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:57.005911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:19:57.152431 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:57.152406    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"]
Jul 17 10:19:57.154901 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:19:57.154869    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf687553_b4df_4672_81eb_c1cffbf4e66f.slice/crio-8f8b74305cdff2e3227c809b385ffa1e39b6b39ca427d69d78eaf99ce1d5008d WatchSource:0}: Error finding container 8f8b74305cdff2e3227c809b385ffa1e39b6b39ca427d69d78eaf99ce1d5008d: Status 404 returned error can't find the container with id 8f8b74305cdff2e3227c809b385ffa1e39b6b39ca427d69d78eaf99ce1d5008d
Jul 17 10:19:57.469398 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:57.469365    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" event={"ID":"cf687553-b4df-4672-81eb-c1cffbf4e66f","Type":"ContainerStarted","Data":"36076394999c17453553ea1988b3efd6369901d049117fc4c265da9b2894cd3f"}
Jul 17 10:19:57.469584 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:57.469422    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" event={"ID":"cf687553-b4df-4672-81eb-c1cffbf4e66f","Type":"ContainerStarted","Data":"8f8b74305cdff2e3227c809b385ffa1e39b6b39ca427d69d78eaf99ce1d5008d"}
Jul 17 10:19:57.485913 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:57.485866    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" podStartSLOduration=1.485852534 podStartE2EDuration="1.485852534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:19:57.485027683 +0000 UTC m=+3181.495151607" watchObservedRunningTime="2026-07-17 10:19:57.485852534 +0000 UTC m=+3181.495976458"
Jul 17 10:19:58.475934 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:58.475894    2556 generic.go:356] "Generic (PLEG): container finished" podID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerID="36076394999c17453553ea1988b3efd6369901d049117fc4c265da9b2894cd3f" exitCode=6
Jul 17 10:19:58.476338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:58.475980    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" event={"ID":"cf687553-b4df-4672-81eb-c1cffbf4e66f","Type":"ContainerDied","Data":"36076394999c17453553ea1988b3efd6369901d049117fc4c265da9b2894cd3f"}
Jul 17 10:19:58.476338 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:58.476151    2556 scope.go:117] "RemoveContainer" containerID="36076394999c17453553ea1988b3efd6369901d049117fc4c265da9b2894cd3f"
Jul 17 10:19:59.482561 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:59.482516    2556 generic.go:356] "Generic (PLEG): container finished" podID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerID="e2dcc6a2c21928d3741624d14e49a8f32be38b611814635c06c88bc823cab879" exitCode=6
Jul 17 10:19:59.483030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:59.482614    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" event={"ID":"cf687553-b4df-4672-81eb-c1cffbf4e66f","Type":"ContainerDied","Data":"e2dcc6a2c21928d3741624d14e49a8f32be38b611814635c06c88bc823cab879"}
Jul 17 10:19:59.483030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:59.482664    2556 scope.go:117] "RemoveContainer" containerID="36076394999c17453553ea1988b3efd6369901d049117fc4c265da9b2894cd3f"
Jul 17 10:19:59.483030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:19:59.482806    2556 scope.go:117] "RemoveContainer" containerID="e2dcc6a2c21928d3741624d14e49a8f32be38b611814635c06c88bc823cab879"
Jul 17 10:19:59.483174 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:19:59.483053    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm_odh-ai-gateway-infra(cf687553-b4df-4672-81eb-c1cffbf4e66f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f"
Jul 17 10:20:00.488692 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:00.488659    2556 scope.go:117] "RemoveContainer" containerID="e2dcc6a2c21928d3741624d14e49a8f32be38b611814635c06c88bc823cab879"
Jul 17 10:20:00.489131 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:20:00.488849    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm_odh-ai-gateway-infra(cf687553-b4df-4672-81eb-c1cffbf4e66f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f"
Jul 17 10:20:02.362508 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.362471    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"]
Jul 17 10:20:02.367016 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.366989    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.370529 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.370503    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"sub-test-model-af80d9-kserve-self-signed-certs\""
Jul 17 10:20:02.370789 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.370503    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"default-dockercfg-s6q2v\""
Jul 17 10:20:02.370917 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.370504    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"openshift-service-ca.crt\""
Jul 17 10:20:02.370917 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.370599    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"kube-root-ca.crt\""
Jul 17 10:20:02.375604 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.375576    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"]
Jul 17 10:20:02.436985 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.436948    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:02.471724 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.471686    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tls-certs\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.471940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.471739    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-model-cache\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.471940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.471800    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfphm\" (UniqueName: \"kubernetes.io/projected/28ee3893-3ece-4e95-b3eb-328e7de4a87d-kube-api-access-zfphm\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.471940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.471919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-dshm\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.472118 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.471957    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-home\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.472118 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.471996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tmp-dir\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.573497 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573439    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-home\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.573726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573512    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tmp-dir\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.573726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573542    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tls-certs\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.573726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573599    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-model-cache\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.573726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573623    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfphm\" (UniqueName: \"kubernetes.io/projected/28ee3893-3ece-4e95-b3eb-328e7de4a87d-kube-api-access-zfphm\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.573726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-dshm\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.574046 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573932    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-home\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.574046 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.573991    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tmp-dir\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.574046 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.574012    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-model-cache\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.576257 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.576232    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-dshm\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.576569 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.576546    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tls-certs\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.582680 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.582650    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfphm\" (UniqueName: \"kubernetes.io/projected/28ee3893-3ece-4e95-b3eb-328e7de4a87d-kube-api-access-zfphm\") pod \"sub-test-model-af80d9-kserve-65d47767ff-4d7dw\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") " pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.678692 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.678646    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:02.818430 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:02.818401    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"]
Jul 17 10:20:02.820988 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:20:02.820962    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28ee3893_3ece_4e95_b3eb_328e7de4a87d.slice/crio-5c055ccbcca4d03b205af5542c118b75c7ab8a6d7819d21421f05d097fe4ce0c WatchSource:0}: Error finding container 5c055ccbcca4d03b205af5542c118b75c7ab8a6d7819d21421f05d097fe4ce0c: Status 404 returned error can't find the container with id 5c055ccbcca4d03b205af5542c118b75c7ab8a6d7819d21421f05d097fe4ce0c
Jul 17 10:20:03.503054 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:03.503014    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw" event={"ID":"28ee3893-3ece-4e95-b3eb-328e7de4a87d","Type":"ContainerStarted","Data":"27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742"}
Jul 17 10:20:03.503460 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:03.503065    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw" event={"ID":"28ee3893-3ece-4e95-b3eb-328e7de4a87d","Type":"ContainerStarted","Data":"5c055ccbcca4d03b205af5542c118b75c7ab8a6d7819d21421f05d097fe4ce0c"}
Jul 17 10:20:03.520230 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:03.520168    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw" podStartSLOduration=1.520149041 podStartE2EDuration="1.520149041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:20:03.519721188 +0000 UTC m=+3187.529845135" watchObservedRunningTime="2026-07-17 10:20:03.520149041 +0000 UTC m=+3187.530272965"
Jul 17 10:20:05.390840 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:05.390763    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:05.491841 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:05.491778    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:07.461242 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:07.461196    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:20:12.490213 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:12.490181    2556 scope.go:117] "RemoveContainer" containerID="e2dcc6a2c21928d3741624d14e49a8f32be38b611814635c06c88bc823cab879"
Jul 17 10:20:12.679720 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:12.679685    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:12.679928 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:12.679744    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:12.692969 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:12.692934    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:13.557806 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:13.557763    2556 generic.go:356] "Generic (PLEG): container finished" podID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerID="75f832778052f4f92195a8c9732c1967d5e86012a0034f35708b5c1d75dd7082" exitCode=6
Jul 17 10:20:13.558307 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:13.557856    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" event={"ID":"cf687553-b4df-4672-81eb-c1cffbf4e66f","Type":"ContainerDied","Data":"75f832778052f4f92195a8c9732c1967d5e86012a0034f35708b5c1d75dd7082"}
Jul 17 10:20:13.558307 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:13.557963    2556 scope.go:117] "RemoveContainer" containerID="e2dcc6a2c21928d3741624d14e49a8f32be38b611814635c06c88bc823cab879"
Jul 17 10:20:13.558390 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:13.558311    2556 scope.go:117] "RemoveContainer" containerID="75f832778052f4f92195a8c9732c1967d5e86012a0034f35708b5c1d75dd7082"
Jul 17 10:20:13.558519 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:20:13.558493    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm_odh-ai-gateway-infra(cf687553-b4df-4672-81eb-c1cffbf4e66f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f"
Jul 17 10:20:13.571474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:13.571447    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:14.583845 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:14.583791    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"]
Jul 17 10:20:14.718351 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:14.718324    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:20:14.780816 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:14.780783    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf687553-b4df-4672-81eb-c1cffbf4e66f-maas-api-service-ca\") pod \"cf687553-b4df-4672-81eb-c1cffbf4e66f\" (UID: \"cf687553-b4df-4672-81eb-c1cffbf4e66f\") "
Jul 17 10:20:14.781142 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:14.781120    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf687553-b4df-4672-81eb-c1cffbf4e66f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf687553-b4df-4672-81eb-c1cffbf4e66f" (UID: "cf687553-b4df-4672-81eb-c1cffbf4e66f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:20:14.881555 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:14.881519    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf687553-b4df-4672-81eb-c1cffbf4e66f-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:15.572059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:15.572028    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"
Jul 17 10:20:15.572259 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:15.572060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm" event={"ID":"cf687553-b4df-4672-81eb-c1cffbf4e66f","Type":"ContainerDied","Data":"8f8b74305cdff2e3227c809b385ffa1e39b6b39ca427d69d78eaf99ce1d5008d"}
Jul 17 10:20:15.572259 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:15.572117    2556 scope.go:117] "RemoveContainer" containerID="75f832778052f4f92195a8c9732c1967d5e86012a0034f35708b5c1d75dd7082"
Jul 17 10:20:15.596271 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:15.596235    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"]
Jul 17 10:20:15.600261 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:15.600227    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qt9lm"]
Jul 17 10:20:16.495258 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:16.495173    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" path="/var/lib/kubelet/pods/cf687553-b4df-4672-81eb-c1cffbf4e66f/volumes"
Jul 17 10:20:21.566389 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:21.566349    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:20:38.415269 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:38.415234    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"]
Jul 17 10:20:38.508140 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:38.508084    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:39.350255 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:39.350197    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"]
Jul 17 10:20:39.372931 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:39.372895    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:43.266640 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:43.266603    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:20:51.067241 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067160    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"]
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067462    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067474    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067483    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067489    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067546    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067554    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.067686 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.067560    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:20:51.070274 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.070257    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.072802 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.072774    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ed096b\"/\"rate-test-model-ed096b-kserve-self-signed-certs\""
Jul 17 10:20:51.079480 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.079456    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"]
Jul 17 10:20:51.154699 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.154661    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:51.192856 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.192786    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tmp-dir\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.193036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.192912    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-dshm\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.193036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.192972    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-model-cache\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.193036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.193001    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp8n2\" (UniqueName: \"kubernetes.io/projected/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-kube-api-access-wp8n2\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.193036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.193032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-home\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.193220 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.193056    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tls-certs\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.293682 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.293633    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-home\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.293682 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.293682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tls-certs\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.293952 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.293731    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tmp-dir\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.293952 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.293770    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-dshm\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.293952 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.293810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-model-cache\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.293952 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.293865    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp8n2\" (UniqueName: \"kubernetes.io/projected/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-kube-api-access-wp8n2\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.294170 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.294148    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-home\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.294229 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.294188    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tmp-dir\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.294229 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.294214    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-model-cache\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.296325 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.296300    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-dshm\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.296620 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.296599    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tls-certs\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.302084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.302058    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp8n2\" (UniqueName: \"kubernetes.io/projected/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-kube-api-access-wp8n2\") pod \"rate-test-model-ed096b-kserve-96f74c7f5-dxbdh\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") " pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.380684 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.380644    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:20:51.523977 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.523949    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"]
Jul 17 10:20:51.526792 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:20:51.526763    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8852ac5_4d06_4b30_8c02_020fdbcd4b85.slice/crio-2bdfd481a74ae021f6f218e2dfbbe51f2f567f389eea0c14ad018661315b8fe2 WatchSource:0}: Error finding container 2bdfd481a74ae021f6f218e2dfbbe51f2f567f389eea0c14ad018661315b8fe2: Status 404 returned error can't find the container with id 2bdfd481a74ae021f6f218e2dfbbe51f2f567f389eea0c14ad018661315b8fe2
Jul 17 10:20:51.747787 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.747688    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh" event={"ID":"b8852ac5-4d06-4b30-8c02-020fdbcd4b85","Type":"ContainerStarted","Data":"ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252"}
Jul 17 10:20:51.747787 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.747737    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh" event={"ID":"b8852ac5-4d06-4b30-8c02-020fdbcd4b85","Type":"ContainerStarted","Data":"2bdfd481a74ae021f6f218e2dfbbe51f2f567f389eea0c14ad018661315b8fe2"}
Jul 17 10:20:51.765188 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:51.765092    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh" podStartSLOduration=0.765063754 podStartE2EDuration="765.063754ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:20:51.7644946 +0000 UTC m=+3235.774618526" watchObservedRunningTime="2026-07-17 10:20:51.765063754 +0000 UTC m=+3235.775187678"
Jul 17 10:20:53.435813 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.435744    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn" podUID="f3ccd25c-a65c-443f-8de4-bad9211a78f5" containerName="main" containerID="cri-o://c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823" gracePeriod=45
Jul 17 10:20:53.692712 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.692647    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:20:53.759170 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.759130    2556 generic.go:356] "Generic (PLEG): container finished" podID="f3ccd25c-a65c-443f-8de4-bad9211a78f5" containerID="c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823" exitCode=0
Jul 17 10:20:53.759344 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.759181    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"
Jul 17 10:20:53.759344 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.759192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn" event={"ID":"f3ccd25c-a65c-443f-8de4-bad9211a78f5","Type":"ContainerDied","Data":"c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823"}
Jul 17 10:20:53.759344 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.759237    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn" event={"ID":"f3ccd25c-a65c-443f-8de4-bad9211a78f5","Type":"ContainerDied","Data":"b6c9051a7f267cec79ed382cb13bd8d74c1e7af6d12b8d73318a04df67e54622"}
Jul 17 10:20:53.759344 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.759255    2556 scope.go:117] "RemoveContainer" containerID="c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823"
Jul 17 10:20:53.767979 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.767956    2556 scope.go:117] "RemoveContainer" containerID="c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823"
Jul 17 10:20:53.768317 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:20:53.768297    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823\": container with ID starting with c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823 not found: ID does not exist" containerID="c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823"
Jul 17 10:20:53.768394 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.768344    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823"} err="failed to get container status \"c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823\": rpc error: code = NotFound desc = could not find container \"c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823\": container with ID starting with c67901eaf5186d56079c71159ec15d57e5256887090858c2e3821ab38dfc8823 not found: ID does not exist"
Jul 17 10:20:53.819492 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819454    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-dshm\") pod \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") "
Jul 17 10:20:53.819654 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819558    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-home\") pod \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") "
Jul 17 10:20:53.819654 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819600    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-model-cache\") pod \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") "
Jul 17 10:20:53.819654 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819637    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tmp-dir\") pod \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") "
Jul 17 10:20:53.820055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819662    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbz7r\" (UniqueName: \"kubernetes.io/projected/f3ccd25c-a65c-443f-8de4-bad9211a78f5-kube-api-access-fbz7r\") pod \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") "
Jul 17 10:20:53.820055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819718    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tls-certs\") pod \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\" (UID: \"f3ccd25c-a65c-443f-8de4-bad9211a78f5\") "
Jul 17 10:20:53.820055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819843    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-home" (OuterVolumeSpecName: "home") pod "f3ccd25c-a65c-443f-8de4-bad9211a78f5" (UID: "f3ccd25c-a65c-443f-8de4-bad9211a78f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:53.820055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819876    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3ccd25c-a65c-443f-8de4-bad9211a78f5" (UID: "f3ccd25c-a65c-443f-8de4-bad9211a78f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:53.820055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819961    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:53.820055 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.819977    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:53.820383 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.820090    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3ccd25c-a65c-443f-8de4-bad9211a78f5" (UID: "f3ccd25c-a65c-443f-8de4-bad9211a78f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:53.822474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.822435    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-dshm" (OuterVolumeSpecName: "dshm") pod "f3ccd25c-a65c-443f-8de4-bad9211a78f5" (UID: "f3ccd25c-a65c-443f-8de4-bad9211a78f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:53.822474 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.822442    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3ccd25c-a65c-443f-8de4-bad9211a78f5" (UID: "f3ccd25c-a65c-443f-8de4-bad9211a78f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:20:53.822661 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.822640    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3ccd25c-a65c-443f-8de4-bad9211a78f5-kube-api-access-fbz7r" (OuterVolumeSpecName: "kube-api-access-fbz7r") pod "f3ccd25c-a65c-443f-8de4-bad9211a78f5" (UID: "f3ccd25c-a65c-443f-8de4-bad9211a78f5"). InnerVolumeSpecName "kube-api-access-fbz7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:20:53.921419 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.921379    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:53.921419 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.921414    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:53.921419 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.921425    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3ccd25c-a65c-443f-8de4-bad9211a78f5-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:53.921687 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:53.921439    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbz7r\" (UniqueName: \"kubernetes.io/projected/f3ccd25c-a65c-443f-8de4-bad9211a78f5-kube-api-access-fbz7r\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:54.084306 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.084263    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"]
Jul 17 10:20:54.087533 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.087501    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/sub-test-model-ed096b-kserve-6564d45867-pqxnn"]
Jul 17 10:20:54.372118 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.372065    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw" podUID="28ee3893-3ece-4e95-b3eb-328e7de4a87d" containerName="main" containerID="cri-o://27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742" gracePeriod=45
Jul 17 10:20:54.497808 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.497775    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3ccd25c-a65c-443f-8de4-bad9211a78f5" path="/var/lib/kubelet/pods/f3ccd25c-a65c-443f-8de4-bad9211a78f5/volumes"
Jul 17 10:20:54.629854 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.629761    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:54.728864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.728830    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tmp-dir\") pod \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") "
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.728894    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tls-certs\") pod \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") "
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.728948    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-model-cache\") pod \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") "
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.728979    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-home\") pod \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") "
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729022    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfphm\" (UniqueName: \"kubernetes.io/projected/28ee3893-3ece-4e95-b3eb-328e7de4a87d-kube-api-access-zfphm\") pod \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") "
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729046    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-dshm\") pod \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\" (UID: \"28ee3893-3ece-4e95-b3eb-328e7de4a87d\") "
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729210    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "28ee3893-3ece-4e95-b3eb-328e7de4a87d" (UID: "28ee3893-3ece-4e95-b3eb-328e7de4a87d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729276    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-model-cache" (OuterVolumeSpecName: "model-cache") pod "28ee3893-3ece-4e95-b3eb-328e7de4a87d" (UID: "28ee3893-3ece-4e95-b3eb-328e7de4a87d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729332    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-home" (OuterVolumeSpecName: "home") pod "28ee3893-3ece-4e95-b3eb-328e7de4a87d" (UID: "28ee3893-3ece-4e95-b3eb-328e7de4a87d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729354    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:54.729412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.729372    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:54.731546 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.731512    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-dshm" (OuterVolumeSpecName: "dshm") pod "28ee3893-3ece-4e95-b3eb-328e7de4a87d" (UID: "28ee3893-3ece-4e95-b3eb-328e7de4a87d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:20:54.731669 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.731523    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28ee3893-3ece-4e95-b3eb-328e7de4a87d-kube-api-access-zfphm" (OuterVolumeSpecName: "kube-api-access-zfphm") pod "28ee3893-3ece-4e95-b3eb-328e7de4a87d" (UID: "28ee3893-3ece-4e95-b3eb-328e7de4a87d"). InnerVolumeSpecName "kube-api-access-zfphm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:20:54.731835 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.731796    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "28ee3893-3ece-4e95-b3eb-328e7de4a87d" (UID: "28ee3893-3ece-4e95-b3eb-328e7de4a87d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:20:54.767543 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.767503    2556 generic.go:356] "Generic (PLEG): container finished" podID="28ee3893-3ece-4e95-b3eb-328e7de4a87d" containerID="27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742" exitCode=0
Jul 17 10:20:54.767750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.767555    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"
Jul 17 10:20:54.767750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.767601    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw" event={"ID":"28ee3893-3ece-4e95-b3eb-328e7de4a87d","Type":"ContainerDied","Data":"27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742"}
Jul 17 10:20:54.767750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.767654    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw" event={"ID":"28ee3893-3ece-4e95-b3eb-328e7de4a87d","Type":"ContainerDied","Data":"5c055ccbcca4d03b205af5542c118b75c7ab8a6d7819d21421f05d097fe4ce0c"}
Jul 17 10:20:54.767750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.767676    2556 scope.go:117] "RemoveContainer" containerID="27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742"
Jul 17 10:20:54.778018 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.777994    2556 scope.go:117] "RemoveContainer" containerID="27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742"
Jul 17 10:20:54.778363 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:20:54.778341    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742\": container with ID starting with 27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742 not found: ID does not exist" containerID="27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742"
Jul 17 10:20:54.778453 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.778377    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742"} err="failed to get container status \"27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742\": rpc error: code = NotFound desc = could not find container \"27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742\": container with ID starting with 27d4ac557d6c8e64b05d41fa4c93838ad652c174b4adec2b693183276f535742 not found: ID does not exist"
Jul 17 10:20:54.789653 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.789607    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"]
Jul 17 10:20:54.791957 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.791928    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/sub-test-model-af80d9-kserve-65d47767ff-4d7dw"]
Jul 17 10:20:54.830070 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.830026    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28ee3893-3ece-4e95-b3eb-328e7de4a87d-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:54.830070 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.830071    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:54.830283 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.830087    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfphm\" (UniqueName: \"kubernetes.io/projected/28ee3893-3ece-4e95-b3eb-328e7de4a87d-kube-api-access-zfphm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:54.830283 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:54.830101    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28ee3893-3ece-4e95-b3eb-328e7de4a87d-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:20:56.496699 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:56.496661    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ee3893-3ece-4e95-b3eb-328e7de4a87d" path="/var/lib/kubelet/pods/28ee3893-3ece-4e95-b3eb-328e7de4a87d/volumes"
Jul 17 10:20:58.015284 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:58.015248    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:20:58.117810 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:20:58.117774    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:21:01.381029 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:01.380975    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:21:01.381029 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:01.381028    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:21:01.393881 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:01.393837    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:21:01.812383 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:01.812287    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:21:04.265122 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:04.265086    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:21:30.128164 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:30.128120    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:21:31.468435 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:31.468401    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:21:36.776196 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776160    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"]
Jul 17 10:21:36.776702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776599    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:21:36.776702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776619    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf687553-b4df-4672-81eb-c1cffbf4e66f" containerName="revoke-keys"
Jul 17 10:21:36.776702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776641    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ee3893-3ece-4e95-b3eb-328e7de4a87d" containerName="main"
Jul 17 10:21:36.776702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776649    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ee3893-3ece-4e95-b3eb-328e7de4a87d" containerName="main"
Jul 17 10:21:36.776702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776670    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3ccd25c-a65c-443f-8de4-bad9211a78f5" containerName="main"
Jul 17 10:21:36.776702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776678    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3ccd25c-a65c-443f-8de4-bad9211a78f5" containerName="main"
Jul 17 10:21:36.777060 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776754    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3ccd25c-a65c-443f-8de4-bad9211a78f5" containerName="main"
Jul 17 10:21:36.777060 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.776770    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ee3893-3ece-4e95-b3eb-328e7de4a87d" containerName="main"
Jul 17 10:21:36.779924 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.779889    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:36.782798 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.782773    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:21:36.782983 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.782772    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:21:36.787138 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.787106    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"]
Jul 17 10:21:36.892534 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.892456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3021c393-f34d-45a4-97a5-510501c94350-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm\" (UID: \"3021c393-f34d-45a4-97a5-510501c94350\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:36.993459 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.993411    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3021c393-f34d-45a4-97a5-510501c94350-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm\" (UID: \"3021c393-f34d-45a4-97a5-510501c94350\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:36.994097 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:36.994074    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3021c393-f34d-45a4-97a5-510501c94350-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm\" (UID: \"3021c393-f34d-45a4-97a5-510501c94350\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:37.093047 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.092954    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:37.240038 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.240005    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"]
Jul 17 10:21:37.242540 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:21:37.242508    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3021c393_f34d_45a4_97a5_510501c94350.slice/crio-c6d246b4570e8b06f4bad0147b932b925c3b343d71bb2de2bafde600b5296bbe WatchSource:0}: Error finding container c6d246b4570e8b06f4bad0147b932b925c3b343d71bb2de2bafde600b5296bbe: Status 404 returned error can't find the container with id c6d246b4570e8b06f4bad0147b932b925c3b343d71bb2de2bafde600b5296bbe
Jul 17 10:21:37.975584 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.975544    2556 generic.go:356] "Generic (PLEG): container finished" podID="3021c393-f34d-45a4-97a5-510501c94350" containerID="23b568573d5b8c1be970a642906dcf273ddd19d659e5c91f82016b15d8d17f3d" exitCode=6
Jul 17 10:21:37.976079 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.975634    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" event={"ID":"3021c393-f34d-45a4-97a5-510501c94350","Type":"ContainerDied","Data":"23b568573d5b8c1be970a642906dcf273ddd19d659e5c91f82016b15d8d17f3d"}
Jul 17 10:21:37.976079 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.975704    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" event={"ID":"3021c393-f34d-45a4-97a5-510501c94350","Type":"ContainerStarted","Data":"c6d246b4570e8b06f4bad0147b932b925c3b343d71bb2de2bafde600b5296bbe"}
Jul 17 10:21:37.976079 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.975842    2556 scope.go:117] "RemoveContainer" containerID="23b568573d5b8c1be970a642906dcf273ddd19d659e5c91f82016b15d8d17f3d"
Jul 17 10:21:37.997164 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:37.997131    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:21:38.060680 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:38.060634    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:21:38.981695 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:38.981654    2556 generic.go:356] "Generic (PLEG): container finished" podID="3021c393-f34d-45a4-97a5-510501c94350" containerID="01bc47f705c7393d6b4395de7899bc8b48121860cb2f0463fd116ea280653584" exitCode=6
Jul 17 10:21:38.982181 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:38.981711    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" event={"ID":"3021c393-f34d-45a4-97a5-510501c94350","Type":"ContainerDied","Data":"01bc47f705c7393d6b4395de7899bc8b48121860cb2f0463fd116ea280653584"}
Jul 17 10:21:38.982181 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:38.981766    2556 scope.go:117] "RemoveContainer" containerID="23b568573d5b8c1be970a642906dcf273ddd19d659e5c91f82016b15d8d17f3d"
Jul 17 10:21:38.982181 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:38.981969    2556 scope.go:117] "RemoveContainer" containerID="01bc47f705c7393d6b4395de7899bc8b48121860cb2f0463fd116ea280653584"
Jul 17 10:21:38.982181 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:21:38.982142    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm_odh-ai-gateway-infra(3021c393-f34d-45a4-97a5-510501c94350)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" podUID="3021c393-f34d-45a4-97a5-510501c94350"
Jul 17 10:21:39.993925 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:39.993891    2556 scope.go:117] "RemoveContainer" containerID="01bc47f705c7393d6b4395de7899bc8b48121860cb2f0463fd116ea280653584"
Jul 17 10:21:39.994389 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:21:39.994153    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm_odh-ai-gateway-infra(3021c393-f34d-45a4-97a5-510501c94350)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" podUID="3021c393-f34d-45a4-97a5-510501c94350"
Jul 17 10:21:40.766448 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.766412    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"]
Jul 17 10:21:40.770989 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.770962    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:40.773354 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.773330    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"kube-root-ca.crt\""
Jul 17 10:21:40.774303 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.774283    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"openshift-service-ca.crt\""
Jul 17 10:21:40.774380 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.774339    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"default-dockercfg-s6q2v\""
Jul 17 10:21:40.774428 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.774403    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-af80d9\"/\"rate-test-model-af80d9-kserve-self-signed-certs\""
Jul 17 10:21:40.778891 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.778370    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"]
Jul 17 10:21:40.790149 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.790106    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:21:40.923750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.923706    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-home\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:40.923750 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.923750    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndzml\" (UniqueName: \"kubernetes.io/projected/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-kube-api-access-ndzml\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:40.924069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.923801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tmp-dir\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:40.924069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.923919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tls-certs\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:40.924069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.923960    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-dshm\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:40.924069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:40.924002    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-model-cache\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.024808 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.024699    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-home\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.024807    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndzml\" (UniqueName: \"kubernetes.io/projected/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-kube-api-access-ndzml\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.024885    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tmp-dir\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.024941    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tls-certs\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.024971    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-dshm\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.025017    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-model-cache\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.025155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-home\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025299 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.025248    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tmp-dir\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.025522 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.025334    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-model-cache\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.027359 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.027339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-dshm\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.027627 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.027610    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tls-certs\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.033808 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.033779    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndzml\" (UniqueName: \"kubernetes.io/projected/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-kube-api-access-ndzml\") pod \"rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") " pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.081716 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.081652    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:41.226107 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:41.226069    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"]
Jul 17 10:21:41.229238 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:21:41.229195    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98381cdd_fa62_4a2b_8f98_2c5d43217ba2.slice/crio-d0d3a972502b4baed3a19e55f961e5041db45953bde8bfe664caeb05c364ecde WatchSource:0}: Error finding container d0d3a972502b4baed3a19e55f961e5041db45953bde8bfe664caeb05c364ecde: Status 404 returned error can't find the container with id d0d3a972502b4baed3a19e55f961e5041db45953bde8bfe664caeb05c364ecde
Jul 17 10:21:42.007758 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:42.007720    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj" event={"ID":"98381cdd-fa62-4a2b-8f98-2c5d43217ba2","Type":"ContainerStarted","Data":"61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7"}
Jul 17 10:21:42.007946 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:42.007770    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj" event={"ID":"98381cdd-fa62-4a2b-8f98-2c5d43217ba2","Type":"ContainerStarted","Data":"d0d3a972502b4baed3a19e55f961e5041db45953bde8bfe664caeb05c364ecde"}
Jul 17 10:21:42.025644 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:42.025592    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj" podStartSLOduration=2.02557509 podStartE2EDuration="2.02557509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:21:42.024190737 +0000 UTC m=+3286.034314663" watchObservedRunningTime="2026-07-17 10:21:42.02557509 +0000 UTC m=+3286.035699014"
Jul 17 10:21:45.570342 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:45.570300    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:21:51.082251 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:51.082201    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:51.082701 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:51.082272    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:51.095634 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:51.095599    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:52.067007 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:52.066975    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:21:54.489884 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:54.489852    2556 scope.go:117] "RemoveContainer" containerID="01bc47f705c7393d6b4395de7899bc8b48121860cb2f0463fd116ea280653584"
Jul 17 10:21:55.071108 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:55.071020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" event={"ID":"3021c393-f34d-45a4-97a5-510501c94350","Type":"ContainerStarted","Data":"244e4b93e5a0f7d55adcf26f6f29afa74d052753d530d9e6909122d26f44069a"}
Jul 17 10:21:55.087327 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:55.087275    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" podStartSLOduration=19.08725692 podStartE2EDuration="19.08725692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:21:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:21:55.0854902 +0000 UTC m=+3299.095614150" watchObservedRunningTime="2026-07-17 10:21:55.08725692 +0000 UTC m=+3299.097380843"
Jul 17 10:21:56.077449 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.077412    2556 generic.go:356] "Generic (PLEG): container finished" podID="3021c393-f34d-45a4-97a5-510501c94350" containerID="244e4b93e5a0f7d55adcf26f6f29afa74d052753d530d9e6909122d26f44069a" exitCode=6
Jul 17 10:21:56.077901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.077485    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" event={"ID":"3021c393-f34d-45a4-97a5-510501c94350","Type":"ContainerDied","Data":"244e4b93e5a0f7d55adcf26f6f29afa74d052753d530d9e6909122d26f44069a"}
Jul 17 10:21:56.077901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.077549    2556 scope.go:117] "RemoveContainer" containerID="01bc47f705c7393d6b4395de7899bc8b48121860cb2f0463fd116ea280653584"
Jul 17 10:21:56.077901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.077872    2556 scope.go:117] "RemoveContainer" containerID="244e4b93e5a0f7d55adcf26f6f29afa74d052753d530d9e6909122d26f44069a"
Jul 17 10:21:56.078129 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:21:56.078103    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm_odh-ai-gateway-infra(3021c393-f34d-45a4-97a5-510501c94350)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" podUID="3021c393-f34d-45a4-97a5-510501c94350"
Jul 17 10:21:56.101523 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.101483    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"]
Jul 17 10:21:56.805435 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.805402    2556 scope.go:117] "RemoveContainer" containerID="191edde92745ad30ade2f58d834d6ab3dd87d3bdb3888d3494cab5734f02e10f"
Jul 17 10:21:56.814124 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:56.814093    2556 scope.go:117] "RemoveContainer" containerID="bb48c5116b92cd971694d686c44edad0b3c3f68467d462b76c1216c698861394"
Jul 17 10:21:57.215620 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:57.215593    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:57.366641 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:57.366538    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3021c393-f34d-45a4-97a5-510501c94350-maas-api-service-ca\") pod \"3021c393-f34d-45a4-97a5-510501c94350\" (UID: \"3021c393-f34d-45a4-97a5-510501c94350\") "
Jul 17 10:21:57.366925 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:57.366904    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3021c393-f34d-45a4-97a5-510501c94350-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3021c393-f34d-45a4-97a5-510501c94350" (UID: "3021c393-f34d-45a4-97a5-510501c94350"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:21:57.467848 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:57.467783    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3021c393-f34d-45a4-97a5-510501c94350-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:21:58.089098 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:58.089067    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"
Jul 17 10:21:58.089098 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:58.089085    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm" event={"ID":"3021c393-f34d-45a4-97a5-510501c94350","Type":"ContainerDied","Data":"c6d246b4570e8b06f4bad0147b932b925c3b343d71bb2de2bafde600b5296bbe"}
Jul 17 10:21:58.089357 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:58.089149    2556 scope.go:117] "RemoveContainer" containerID="244e4b93e5a0f7d55adcf26f6f29afa74d052753d530d9e6909122d26f44069a"
Jul 17 10:21:58.109415 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:58.109381    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"]
Jul 17 10:21:58.115236 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:58.115209    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-sgwwm"]
Jul 17 10:21:58.495797 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:21:58.495756    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3021c393-f34d-45a4-97a5-510501c94350" path="/var/lib/kubelet/pods/3021c393-f34d-45a4-97a5-510501c94350/volumes"
Jul 17 10:22:07.365660 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:07.365617    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:22:23.257802 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:23.257760    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:22:31.452730 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:31.452687    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"]
Jul 17 10:22:32.100079 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:32.100029    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:22:32.576789 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:32.576748    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"]
Jul 17 10:22:32.597728 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:32.597688    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:22:33.564272 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564235    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-140042772875952"]
Jul 17 10:22:33.564559 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564546    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:33.564625 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564561    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:33.564625 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564574    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:33.564625 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564580    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:33.564737 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564638    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:33.564737 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.564647    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:33.567830 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.567798    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:33.570412 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.570391    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vs5dz\""
Jul 17 10:22:33.574384 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.574348    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772875952"]
Jul 17 10:22:33.683713 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.683672    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnc5q\" (UniqueName: \"kubernetes.io/projected/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88-kube-api-access-vnc5q\") pod \"test-curl-7411-140042772875952\" (UID: \"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88\") " pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:33.784236 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.784198    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnc5q\" (UniqueName: \"kubernetes.io/projected/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88-kube-api-access-vnc5q\") pod \"test-curl-7411-140042772875952\" (UID: \"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88\") " pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:33.799184 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.799144    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnc5q\" (UniqueName: \"kubernetes.io/projected/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88-kube-api-access-vnc5q\") pod \"test-curl-7411-140042772875952\" (UID: \"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88\") " pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:33.878322 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:33.878291    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:34.016437 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:34.016309    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772875952"]
Jul 17 10:22:34.019309 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:34.019270    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76f9bbe1_3d1f_4804_b05a_8d580f7c0a88.slice/crio-0b4fc432ccf91ec2f8cdedb463a4a0a21b43a51ef4c4c3ceda35d24f5e02874f WatchSource:0}: Error finding container 0b4fc432ccf91ec2f8cdedb463a4a0a21b43a51ef4c4c3ceda35d24f5e02874f: Status 404 returned error can't find the container with id 0b4fc432ccf91ec2f8cdedb463a4a0a21b43a51ef4c4c3ceda35d24f5e02874f
Jul 17 10:22:34.020642 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:34.020618    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:22:34.259879 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:34.259691    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772875952" event={"ID":"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88","Type":"ContainerStarted","Data":"0b4fc432ccf91ec2f8cdedb463a4a0a21b43a51ef4c4c3ceda35d24f5e02874f"}
Jul 17 10:22:35.265886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:35.265847    2556 generic.go:356] "Generic (PLEG): container finished" podID="76f9bbe1-3d1f-4804-b05a-8d580f7c0a88" containerID="16d222859097df5e04ac4a57e448a156f379eb9781b41af37cefada4ec40cc10" exitCode=0
Jul 17 10:22:35.266308 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:35.265901    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772875952" event={"ID":"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88","Type":"ContainerDied","Data":"16d222859097df5e04ac4a57e448a156f379eb9781b41af37cefada4ec40cc10"}
Jul 17 10:22:36.407216 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.407192    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:36.441889 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.441860    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-140042772875952_76f9bbe1-3d1f-4804-b05a-8d580f7c0a88/test-curl-7411-140042772875952/0.log"
Jul 17 10:22:36.468522 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.468478    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-140042772875952"]
Jul 17 10:22:36.472359 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.472331    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-140042772875952"]
Jul 17 10:22:36.509990 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.509960    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnc5q\" (UniqueName: \"kubernetes.io/projected/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88-kube-api-access-vnc5q\") pod \"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88\" (UID: \"76f9bbe1-3d1f-4804-b05a-8d580f7c0a88\") "
Jul 17 10:22:36.512395 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.512369    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88-kube-api-access-vnc5q" (OuterVolumeSpecName: "kube-api-access-vnc5q") pod "76f9bbe1-3d1f-4804-b05a-8d580f7c0a88" (UID: "76f9bbe1-3d1f-4804-b05a-8d580f7c0a88"). InnerVolumeSpecName "kube-api-access-vnc5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:36.598753 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.598664    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-140042772763952"]
Jul 17 10:22:36.599013 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.598999    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:36.599059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.599015    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:36.599059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.599029    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f9bbe1-3d1f-4804-b05a-8d580f7c0a88" containerName="test-curl-7411-140042772875952"
Jul 17 10:22:36.599059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.599035    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f9bbe1-3d1f-4804-b05a-8d580f7c0a88" containerName="test-curl-7411-140042772875952"
Jul 17 10:22:36.599156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.599089    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f9bbe1-3d1f-4804-b05a-8d580f7c0a88" containerName="test-curl-7411-140042772875952"
Jul 17 10:22:36.599156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.599097    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3021c393-f34d-45a4-97a5-510501c94350" containerName="revoke-keys"
Jul 17 10:22:36.602197 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.602171    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:36.608382 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.608351    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772763952"]
Jul 17 10:22:36.611175 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.611147    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnc5q\" (UniqueName: \"kubernetes.io/projected/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88-kube-api-access-vnc5q\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:36.712577 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.712542    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpb68\" (UniqueName: \"kubernetes.io/projected/5037c644-3974-443a-b549-6679e7eca6f2-kube-api-access-kpb68\") pod \"test-curl-7411-140042772763952\" (UID: \"5037c644-3974-443a-b549-6679e7eca6f2\") " pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:36.813348 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.813306    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpb68\" (UniqueName: \"kubernetes.io/projected/5037c644-3974-443a-b549-6679e7eca6f2-kube-api-access-kpb68\") pod \"test-curl-7411-140042772763952\" (UID: \"5037c644-3974-443a-b549-6679e7eca6f2\") " pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:36.822765 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.822726    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpb68\" (UniqueName: \"kubernetes.io/projected/5037c644-3974-443a-b549-6679e7eca6f2-kube-api-access-kpb68\") pod \"test-curl-7411-140042772763952\" (UID: \"5037c644-3974-443a-b549-6679e7eca6f2\") " pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:36.913978 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:36.913932    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:37.061154 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:37.061118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772763952"]
Jul 17 10:22:37.064178 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:37.064143    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5037c644_3974_443a_b549_6679e7eca6f2.slice/crio-49c44cb4486ae13ee7d8c031725bb9eea00d7cc2a9d65de166cf1b3b72bd79aa WatchSource:0}: Error finding container 49c44cb4486ae13ee7d8c031725bb9eea00d7cc2a9d65de166cf1b3b72bd79aa: Status 404 returned error can't find the container with id 49c44cb4486ae13ee7d8c031725bb9eea00d7cc2a9d65de166cf1b3b72bd79aa
Jul 17 10:22:37.276159 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:37.276071    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772875952"
Jul 17 10:22:37.276159 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:37.276123    2556 scope.go:117] "RemoveContainer" containerID="16d222859097df5e04ac4a57e448a156f379eb9781b41af37cefada4ec40cc10"
Jul 17 10:22:37.277594 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:37.277379    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772763952" event={"ID":"5037c644-3974-443a-b549-6679e7eca6f2","Type":"ContainerStarted","Data":"49c44cb4486ae13ee7d8c031725bb9eea00d7cc2a9d65de166cf1b3b72bd79aa"}
Jul 17 10:22:38.284026 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:38.283983    2556 generic.go:356] "Generic (PLEG): container finished" podID="5037c644-3974-443a-b549-6679e7eca6f2" containerID="e8a3c6553458308ed97815b8b1a393650858ad859cc7b0224d5c4e066f339cc9" exitCode=0
Jul 17 10:22:38.284581 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:38.284064    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772763952" event={"ID":"5037c644-3974-443a-b549-6679e7eca6f2","Type":"ContainerDied","Data":"e8a3c6553458308ed97815b8b1a393650858ad859cc7b0224d5c4e066f339cc9"}
Jul 17 10:22:38.496315 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:38.496274    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f9bbe1-3d1f-4804-b05a-8d580f7c0a88" path="/var/lib/kubelet/pods/76f9bbe1-3d1f-4804-b05a-8d580f7c0a88/volumes"
Jul 17 10:22:39.362066 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.362032    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:22:39.424901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.424866    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:39.445629 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.445601    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-140042772763952_5037c644-3974-443a-b549-6679e7eca6f2/test-curl-7411-140042772763952/0.log"
Jul 17 10:22:39.470215 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.470180    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-140042772763952"]
Jul 17 10:22:39.476119 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.476089    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-140042772763952"]
Jul 17 10:22:39.538059 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.538024    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpb68\" (UniqueName: \"kubernetes.io/projected/5037c644-3974-443a-b549-6679e7eca6f2-kube-api-access-kpb68\") pod \"5037c644-3974-443a-b549-6679e7eca6f2\" (UID: \"5037c644-3974-443a-b549-6679e7eca6f2\") "
Jul 17 10:22:39.540428 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.540352    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5037c644-3974-443a-b549-6679e7eca6f2-kube-api-access-kpb68" (OuterVolumeSpecName: "kube-api-access-kpb68") pod "5037c644-3974-443a-b549-6679e7eca6f2" (UID: "5037c644-3974-443a-b549-6679e7eca6f2"). InnerVolumeSpecName "kube-api-access-kpb68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:39.638966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.638926    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpb68\" (UniqueName: \"kubernetes.io/projected/5037c644-3974-443a-b549-6679e7eca6f2-kube-api-access-kpb68\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:39.640399 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.640376    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:39.640688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.640674    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5037c644-3974-443a-b549-6679e7eca6f2" containerName="test-curl-7411-140042772763952"
Jul 17 10:22:39.640727 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.640690    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5037c644-3974-443a-b549-6679e7eca6f2" containerName="test-curl-7411-140042772763952"
Jul 17 10:22:39.640761 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.640753    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5037c644-3974-443a-b549-6679e7eca6f2" containerName="test-curl-7411-140042772763952"
Jul 17 10:22:39.643706 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.643686    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:39.649594 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.649559    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:39.739673 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.739635    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbrdx\" (UniqueName: \"kubernetes.io/projected/97bdba7c-b634-4d74-af75-64d3a84e4adb-kube-api-access-dbrdx\") pod \"test-curl-7411-140042772954832\" (UID: \"97bdba7c-b634-4d74-af75-64d3a84e4adb\") " pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:39.841085 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.840981    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbrdx\" (UniqueName: \"kubernetes.io/projected/97bdba7c-b634-4d74-af75-64d3a84e4adb-kube-api-access-dbrdx\") pod \"test-curl-7411-140042772954832\" (UID: \"97bdba7c-b634-4d74-af75-64d3a84e4adb\") " pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:39.849571 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.849539    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbrdx\" (UniqueName: \"kubernetes.io/projected/97bdba7c-b634-4d74-af75-64d3a84e4adb-kube-api-access-dbrdx\") pod \"test-curl-7411-140042772954832\" (UID: \"97bdba7c-b634-4d74-af75-64d3a84e4adb\") " pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:39.953557 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:39.953509    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:40.094410 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:40.094380    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:40.096703 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:40.096676    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97bdba7c_b634_4d74_af75_64d3a84e4adb.slice/crio-6817b341ef8704ffc4755b6daf44d608e8baa309b687323c12f90a01f0a345e7 WatchSource:0}: Error finding container 6817b341ef8704ffc4755b6daf44d608e8baa309b687323c12f90a01f0a345e7: Status 404 returned error can't find the container with id 6817b341ef8704ffc4755b6daf44d608e8baa309b687323c12f90a01f0a345e7
Jul 17 10:22:40.296225 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:40.296191    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772954832" event={"ID":"97bdba7c-b634-4d74-af75-64d3a84e4adb","Type":"ContainerStarted","Data":"6817b341ef8704ffc4755b6daf44d608e8baa309b687323c12f90a01f0a345e7"}
Jul 17 10:22:40.297586 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:40.297561    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49c44cb4486ae13ee7d8c031725bb9eea00d7cc2a9d65de166cf1b3b72bd79aa"
Jul 17 10:22:40.297732 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:40.297601    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:40.309079 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:40.309033    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-140042772763952\" is forbidden: User \"system:node:ip-10-0-141-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-244.ec2.internal' and this object" podUID="5037c644-3974-443a-b549-6679e7eca6f2" pod="opendatahub/test-curl-7411-140042772763952"
Jul 17 10:22:40.495955 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:40.495921    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5037c644-3974-443a-b549-6679e7eca6f2" path="/var/lib/kubelet/pods/5037c644-3974-443a-b549-6679e7eca6f2/volumes"
Jul 17 10:22:41.304266 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:41.304228    2556 generic.go:356] "Generic (PLEG): container finished" podID="97bdba7c-b634-4d74-af75-64d3a84e4adb" containerID="349863ab0022c5b199446403fcf1461d839151611902978272bb1d9989bc2520" exitCode=0
Jul 17 10:22:41.304445 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:41.304320    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772954832" event={"ID":"97bdba7c-b634-4d74-af75-64d3a84e4adb","Type":"ContainerDied","Data":"349863ab0022c5b199446403fcf1461d839151611902978272bb1d9989bc2520"}
Jul 17 10:22:42.450604 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.450568    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:42.472281 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.472247    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-140042772954832_97bdba7c-b634-4d74-af75-64d3a84e4adb/test-curl-7411-140042772954832/0.log"
Jul 17 10:22:42.497796 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.497761    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:42.501619 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.501588    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:42.568277 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.568186    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbrdx\" (UniqueName: \"kubernetes.io/projected/97bdba7c-b634-4d74-af75-64d3a84e4adb-kube-api-access-dbrdx\") pod \"97bdba7c-b634-4d74-af75-64d3a84e4adb\" (UID: \"97bdba7c-b634-4d74-af75-64d3a84e4adb\") "
Jul 17 10:22:42.570555 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.570523    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97bdba7c-b634-4d74-af75-64d3a84e4adb-kube-api-access-dbrdx" (OuterVolumeSpecName: "kube-api-access-dbrdx") pod "97bdba7c-b634-4d74-af75-64d3a84e4adb" (UID: "97bdba7c-b634-4d74-af75-64d3a84e4adb"). InnerVolumeSpecName "kube-api-access-dbrdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:42.623865 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.623815    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:42.624169 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.624155    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bdba7c-b634-4d74-af75-64d3a84e4adb" containerName="test-curl-7411-140042772954832"
Jul 17 10:22:42.624214 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.624171    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bdba7c-b634-4d74-af75-64d3a84e4adb" containerName="test-curl-7411-140042772954832"
Jul 17 10:22:42.624248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.624219    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="97bdba7c-b634-4d74-af75-64d3a84e4adb" containerName="test-curl-7411-140042772954832"
Jul 17 10:22:42.627036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.627014    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:42.634866 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.634837    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:42.668922 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.668877    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbrdx\" (UniqueName: \"kubernetes.io/projected/97bdba7c-b634-4d74-af75-64d3a84e4adb-kube-api-access-dbrdx\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:42.769441 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.769392    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2phf\" (UniqueName: \"kubernetes.io/projected/3c02cf4d-b61a-4612-b576-093a60975b15-kube-api-access-p2phf\") pod \"test-curl-7411-140042772954832\" (UID: \"3c02cf4d-b61a-4612-b576-093a60975b15\") " pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:42.870534 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.870498    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2phf\" (UniqueName: \"kubernetes.io/projected/3c02cf4d-b61a-4612-b576-093a60975b15-kube-api-access-p2phf\") pod \"test-curl-7411-140042772954832\" (UID: \"3c02cf4d-b61a-4612-b576-093a60975b15\") " pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:42.879108 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.879072    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2phf\" (UniqueName: \"kubernetes.io/projected/3c02cf4d-b61a-4612-b576-093a60975b15-kube-api-access-p2phf\") pod \"test-curl-7411-140042772954832\" (UID: \"3c02cf4d-b61a-4612-b576-093a60975b15\") " pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:42.937354 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:42.937310    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:43.081798 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:43.081763    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:43.085353 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:43.085325    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c02cf4d_b61a_4612_b576_093a60975b15.slice/crio-fc067ca519777e3b3662e24525ad8a3c3f3abe976bed6313521a3d5c3295ffb2 WatchSource:0}: Error finding container fc067ca519777e3b3662e24525ad8a3c3f3abe976bed6313521a3d5c3295ffb2: Status 404 returned error can't find the container with id fc067ca519777e3b3662e24525ad8a3c3f3abe976bed6313521a3d5c3295ffb2
Jul 17 10:22:43.314501 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:43.314458    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772954832" event={"ID":"3c02cf4d-b61a-4612-b576-093a60975b15","Type":"ContainerStarted","Data":"fc067ca519777e3b3662e24525ad8a3c3f3abe976bed6313521a3d5c3295ffb2"}
Jul 17 10:22:43.315654 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:43.315633    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6817b341ef8704ffc4755b6daf44d608e8baa309b687323c12f90a01f0a345e7"
Jul 17 10:22:43.315780 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:43.315668    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:43.327057 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:43.327020    2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7411-140042772954832" oldPodUID="97bdba7c-b634-4d74-af75-64d3a84e4adb" podUID="3c02cf4d-b61a-4612-b576-093a60975b15"
Jul 17 10:22:44.322068 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:44.322034    2556 generic.go:356] "Generic (PLEG): container finished" podID="3c02cf4d-b61a-4612-b576-093a60975b15" containerID="4f73b2e1d16b66d67e495e81667597103d8129af76def4982baf7b87fcde76d4" exitCode=0
Jul 17 10:22:44.322448 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:44.322109    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772954832" event={"ID":"3c02cf4d-b61a-4612-b576-093a60975b15","Type":"ContainerDied","Data":"4f73b2e1d16b66d67e495e81667597103d8129af76def4982baf7b87fcde76d4"}
Jul 17 10:22:44.495150 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:44.495115    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97bdba7c-b634-4d74-af75-64d3a84e4adb" path="/var/lib/kubelet/pods/97bdba7c-b634-4d74-af75-64d3a84e4adb/volumes"
Jul 17 10:22:45.478253 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.478226    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:45.498301 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.498275    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-140042772954832_3c02cf4d-b61a-4612-b576-093a60975b15/test-curl-7411-140042772954832/0.log"
Jul 17 10:22:45.519918 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.519878    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:45.523679 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.523651    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-140042772954832"]
Jul 17 10:22:45.591476 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.591378    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2phf\" (UniqueName: \"kubernetes.io/projected/3c02cf4d-b61a-4612-b576-093a60975b15-kube-api-access-p2phf\") pod \"3c02cf4d-b61a-4612-b576-093a60975b15\" (UID: \"3c02cf4d-b61a-4612-b576-093a60975b15\") "
Jul 17 10:22:45.593641 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.593615    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c02cf4d-b61a-4612-b576-093a60975b15-kube-api-access-p2phf" (OuterVolumeSpecName: "kube-api-access-p2phf") pod "3c02cf4d-b61a-4612-b576-093a60975b15" (UID: "3c02cf4d-b61a-4612-b576-093a60975b15"). InnerVolumeSpecName "kube-api-access-p2phf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:45.646866 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.646809    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:45.647119 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.647106    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c02cf4d-b61a-4612-b576-093a60975b15" containerName="test-curl-7411-140042772954832"
Jul 17 10:22:45.647163 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.647120    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c02cf4d-b61a-4612-b576-093a60975b15" containerName="test-curl-7411-140042772954832"
Jul 17 10:22:45.647219 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.647209    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c02cf4d-b61a-4612-b576-093a60975b15" containerName="test-curl-7411-140042772954832"
Jul 17 10:22:45.650018 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.649996    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:45.654661 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.654628    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:45.692565 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.692529    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2phf\" (UniqueName: \"kubernetes.io/projected/3c02cf4d-b61a-4612-b576-093a60975b15-kube-api-access-p2phf\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:45.793132 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.793098    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbv5\" (UniqueName: \"kubernetes.io/projected/7f19ac6d-6cbb-412b-8204-2acb4135c802-kube-api-access-jjbv5\") pod \"test-curl-7411-140042772919040\" (UID: \"7f19ac6d-6cbb-412b-8204-2acb4135c802\") " pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:45.893739 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.893695    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbv5\" (UniqueName: \"kubernetes.io/projected/7f19ac6d-6cbb-412b-8204-2acb4135c802-kube-api-access-jjbv5\") pod \"test-curl-7411-140042772919040\" (UID: \"7f19ac6d-6cbb-412b-8204-2acb4135c802\") " pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:45.903071 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.903040    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbv5\" (UniqueName: \"kubernetes.io/projected/7f19ac6d-6cbb-412b-8204-2acb4135c802-kube-api-access-jjbv5\") pod \"test-curl-7411-140042772919040\" (UID: \"7f19ac6d-6cbb-412b-8204-2acb4135c802\") " pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:45.961253 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:45.961186    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:46.102086 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.102061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:46.104534 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:46.104507    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f19ac6d_6cbb_412b_8204_2acb4135c802.slice/crio-31749fda7f939230a1dbda398c0e9ba2aed6acec22a6e9db058b8318bce01d8e WatchSource:0}: Error finding container 31749fda7f939230a1dbda398c0e9ba2aed6acec22a6e9db058b8318bce01d8e: Status 404 returned error can't find the container with id 31749fda7f939230a1dbda398c0e9ba2aed6acec22a6e9db058b8318bce01d8e
Jul 17 10:22:46.332345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.332236    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772919040" event={"ID":"7f19ac6d-6cbb-412b-8204-2acb4135c802","Type":"ContainerStarted","Data":"31749fda7f939230a1dbda398c0e9ba2aed6acec22a6e9db058b8318bce01d8e"}
Jul 17 10:22:46.333530 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.333509    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:46.333641 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.333516    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc067ca519777e3b3662e24525ad8a3c3f3abe976bed6313521a3d5c3295ffb2"
Jul 17 10:22:46.343796 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:46.343764    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-140042772954832\" is forbidden: User \"system:node:ip-10-0-141-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-244.ec2.internal' and this object" podUID="3c02cf4d-b61a-4612-b576-093a60975b15" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:46.473129 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.473092    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh" podUID="b8852ac5-4d06-4b30-8c02-020fdbcd4b85" containerName="main" containerID="cri-o://ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252" gracePeriod=45
Jul 17 10:22:46.493605 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:46.493570    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-140042772954832\" is forbidden: User \"system:node:ip-10-0-141-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-244.ec2.internal' and this object" podUID="3c02cf4d-b61a-4612-b576-093a60975b15" pod="opendatahub/test-curl-7411-140042772954832"
Jul 17 10:22:46.496039 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.496013    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c02cf4d-b61a-4612-b576-093a60975b15" path="/var/lib/kubelet/pods/3c02cf4d-b61a-4612-b576-093a60975b15/volumes"
Jul 17 10:22:46.747640 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.747615    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:22:46.901914 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.901876    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-dshm\") pod \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") "
Jul 17 10:22:46.901914 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.901923    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-home\") pod \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") "
Jul 17 10:22:46.902156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.901950    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tmp-dir\") pod \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") "
Jul 17 10:22:46.902156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.901988    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp8n2\" (UniqueName: \"kubernetes.io/projected/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-kube-api-access-wp8n2\") pod \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") "
Jul 17 10:22:46.902156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.902007    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tls-certs\") pod \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") "
Jul 17 10:22:46.902156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.902023    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-model-cache\") pod \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\" (UID: \"b8852ac5-4d06-4b30-8c02-020fdbcd4b85\") "
Jul 17 10:22:46.902361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.902201    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-home" (OuterVolumeSpecName: "home") pod "b8852ac5-4d06-4b30-8c02-020fdbcd4b85" (UID: "b8852ac5-4d06-4b30-8c02-020fdbcd4b85"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:46.902361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.902296    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:46.902459 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.902322    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b8852ac5-4d06-4b30-8c02-020fdbcd4b85" (UID: "b8852ac5-4d06-4b30-8c02-020fdbcd4b85"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:46.902459 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.902417    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-model-cache" (OuterVolumeSpecName: "model-cache") pod "b8852ac5-4d06-4b30-8c02-020fdbcd4b85" (UID: "b8852ac5-4d06-4b30-8c02-020fdbcd4b85"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:46.904345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.904310    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-dshm" (OuterVolumeSpecName: "dshm") pod "b8852ac5-4d06-4b30-8c02-020fdbcd4b85" (UID: "b8852ac5-4d06-4b30-8c02-020fdbcd4b85"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:46.904345 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.904325    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b8852ac5-4d06-4b30-8c02-020fdbcd4b85" (UID: "b8852ac5-4d06-4b30-8c02-020fdbcd4b85"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:22:46.904530 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:46.904345    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-kube-api-access-wp8n2" (OuterVolumeSpecName: "kube-api-access-wp8n2") pod "b8852ac5-4d06-4b30-8c02-020fdbcd4b85" (UID: "b8852ac5-4d06-4b30-8c02-020fdbcd4b85"). InnerVolumeSpecName "kube-api-access-wp8n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:47.003265 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.003227    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:47.003393 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.003262    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:47.003393 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.003289    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp8n2\" (UniqueName: \"kubernetes.io/projected/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-kube-api-access-wp8n2\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:47.003393 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.003297    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:47.003393 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.003305    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8852ac5-4d06-4b30-8c02-020fdbcd4b85-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:47.339593 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.339498    2556 generic.go:356] "Generic (PLEG): container finished" podID="7f19ac6d-6cbb-412b-8204-2acb4135c802" containerID="a0e79e952b03dfb3b40855f8169bb0076491b897f5535c73653359ce734be704" exitCode=0
Jul 17 10:22:47.339593 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.339585    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772919040" event={"ID":"7f19ac6d-6cbb-412b-8204-2acb4135c802","Type":"ContainerDied","Data":"a0e79e952b03dfb3b40855f8169bb0076491b897f5535c73653359ce734be704"}
Jul 17 10:22:47.340935 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.340912    2556 generic.go:356] "Generic (PLEG): container finished" podID="b8852ac5-4d06-4b30-8c02-020fdbcd4b85" containerID="ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252" exitCode=0
Jul 17 10:22:47.341044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.340937    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh" event={"ID":"b8852ac5-4d06-4b30-8c02-020fdbcd4b85","Type":"ContainerDied","Data":"ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252"}
Jul 17 10:22:47.341044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.340963    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"
Jul 17 10:22:47.341044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.340978    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh" event={"ID":"b8852ac5-4d06-4b30-8c02-020fdbcd4b85","Type":"ContainerDied","Data":"2bdfd481a74ae021f6f218e2dfbbe51f2f567f389eea0c14ad018661315b8fe2"}
Jul 17 10:22:47.341044 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.340997    2556 scope.go:117] "RemoveContainer" containerID="ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252"
Jul 17 10:22:47.350134 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.350113    2556 scope.go:117] "RemoveContainer" containerID="ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252"
Jul 17 10:22:47.350440 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:47.350414    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252\": container with ID starting with ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252 not found: ID does not exist" containerID="ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252"
Jul 17 10:22:47.350484 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.350451    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252"} err="failed to get container status \"ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252\": rpc error: code = NotFound desc = could not find container \"ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252\": container with ID starting with ab3d4e63c08f1fec0cb301d0894ac8ccf846aa98bbe4b0fbfeff87a9338bd252 not found: ID does not exist"
Jul 17 10:22:47.365255 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.365218    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"]
Jul 17 10:22:47.368922 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.368891    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ed096b/rate-test-model-ed096b-kserve-96f74c7f5-dxbdh"]
Jul 17 10:22:47.601534 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.601438    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj" podUID="98381cdd-fa62-4a2b-8f98-2c5d43217ba2" containerName="main" containerID="cri-o://61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7" gracePeriod=45
Jul 17 10:22:47.859378 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:47.859312    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:22:48.012589 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012548    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-home\") pod \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") "
Jul 17 10:22:48.012768 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012596    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-dshm\") pod \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") "
Jul 17 10:22:48.012768 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012633    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tls-certs\") pod \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") "
Jul 17 10:22:48.012768 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012656    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tmp-dir\") pod \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") "
Jul 17 10:22:48.012768 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012743    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndzml\" (UniqueName: \"kubernetes.io/projected/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-kube-api-access-ndzml\") pod \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") "
Jul 17 10:22:48.013009 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012769    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-model-cache\") pod \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\" (UID: \"98381cdd-fa62-4a2b-8f98-2c5d43217ba2\") "
Jul 17 10:22:48.013009 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.012919    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-home" (OuterVolumeSpecName: "home") pod "98381cdd-fa62-4a2b-8f98-2c5d43217ba2" (UID: "98381cdd-fa62-4a2b-8f98-2c5d43217ba2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:48.013113 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.013022    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-home\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.013113 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.013095    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-model-cache" (OuterVolumeSpecName: "model-cache") pod "98381cdd-fa62-4a2b-8f98-2c5d43217ba2" (UID: "98381cdd-fa62-4a2b-8f98-2c5d43217ba2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:48.013193 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.013109    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98381cdd-fa62-4a2b-8f98-2c5d43217ba2" (UID: "98381cdd-fa62-4a2b-8f98-2c5d43217ba2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:48.015123 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.015099    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-dshm" (OuterVolumeSpecName: "dshm") pod "98381cdd-fa62-4a2b-8f98-2c5d43217ba2" (UID: "98381cdd-fa62-4a2b-8f98-2c5d43217ba2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:22:48.015123 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.015103    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-kube-api-access-ndzml" (OuterVolumeSpecName: "kube-api-access-ndzml") pod "98381cdd-fa62-4a2b-8f98-2c5d43217ba2" (UID: "98381cdd-fa62-4a2b-8f98-2c5d43217ba2"). InnerVolumeSpecName "kube-api-access-ndzml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:48.015248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.015172    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98381cdd-fa62-4a2b-8f98-2c5d43217ba2" (UID: "98381cdd-fa62-4a2b-8f98-2c5d43217ba2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:22:48.113617 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.113525    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-dshm\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.113617 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.113558    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tls-certs\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.113617 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.113568    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-tmp-dir\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.113617 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.113576    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndzml\" (UniqueName: \"kubernetes.io/projected/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-kube-api-access-ndzml\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.113617 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.113584    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98381cdd-fa62-4a2b-8f98-2c5d43217ba2-model-cache\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.347481 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.347436    2556 generic.go:356] "Generic (PLEG): container finished" podID="98381cdd-fa62-4a2b-8f98-2c5d43217ba2" containerID="61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7" exitCode=0
Jul 17 10:22:48.347481 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.347479    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"
Jul 17 10:22:48.347726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.347526    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj" event={"ID":"98381cdd-fa62-4a2b-8f98-2c5d43217ba2","Type":"ContainerDied","Data":"61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7"}
Jul 17 10:22:48.347726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.347567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj" event={"ID":"98381cdd-fa62-4a2b-8f98-2c5d43217ba2","Type":"ContainerDied","Data":"d0d3a972502b4baed3a19e55f961e5041db45953bde8bfe664caeb05c364ecde"}
Jul 17 10:22:48.347726 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.347591    2556 scope.go:117] "RemoveContainer" containerID="61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7"
Jul 17 10:22:48.357607 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.357588    2556 scope.go:117] "RemoveContainer" containerID="61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7"
Jul 17 10:22:48.357955 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:48.357933    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7\": container with ID starting with 61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7 not found: ID does not exist" containerID="61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7"
Jul 17 10:22:48.358019 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.357965    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7"} err="failed to get container status \"61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7\": rpc error: code = NotFound desc = could not find container \"61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7\": container with ID starting with 61345e4f72ff17cfde8d1f524b07272a8b6e0bb733717292018994aea12363e7 not found: ID does not exist"
Jul 17 10:22:48.370166 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.370135    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"]
Jul 17 10:22:48.375336 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.375296    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-af80d9/rate-test-model-af80d9-kserve-7dc6cbcf79-d4qhj"]
Jul 17 10:22:48.492584 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.492556    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:48.496702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.496669    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98381cdd-fa62-4a2b-8f98-2c5d43217ba2" path="/var/lib/kubelet/pods/98381cdd-fa62-4a2b-8f98-2c5d43217ba2/volumes"
Jul 17 10:22:48.497278 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.497253    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8852ac5-4d06-4b30-8c02-020fdbcd4b85" path="/var/lib/kubelet/pods/b8852ac5-4d06-4b30-8c02-020fdbcd4b85/volumes"
Jul 17 10:22:48.513816 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.513784    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-140042772919040_7f19ac6d-6cbb-412b-8204-2acb4135c802/test-curl-7411-140042772919040/0.log"
Jul 17 10:22:48.536794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.536756    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:48.544218 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.544179    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:48.616315 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.616276    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjbv5\" (UniqueName: \"kubernetes.io/projected/7f19ac6d-6cbb-412b-8204-2acb4135c802-kube-api-access-jjbv5\") pod \"7f19ac6d-6cbb-412b-8204-2acb4135c802\" (UID: \"7f19ac6d-6cbb-412b-8204-2acb4135c802\") "
Jul 17 10:22:48.619316 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.619274    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f19ac6d-6cbb-412b-8204-2acb4135c802-kube-api-access-jjbv5" (OuterVolumeSpecName: "kube-api-access-jjbv5") pod "7f19ac6d-6cbb-412b-8204-2acb4135c802" (UID: "7f19ac6d-6cbb-412b-8204-2acb4135c802"). InnerVolumeSpecName "kube-api-access-jjbv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:48.672490 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672439    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:48.672753 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672737    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98381cdd-fa62-4a2b-8f98-2c5d43217ba2" containerName="main"
Jul 17 10:22:48.672794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672755    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="98381cdd-fa62-4a2b-8f98-2c5d43217ba2" containerName="main"
Jul 17 10:22:48.672794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672768    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f19ac6d-6cbb-412b-8204-2acb4135c802" containerName="test-curl-7411-140042772919040"
Jul 17 10:22:48.672794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672774    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f19ac6d-6cbb-412b-8204-2acb4135c802" containerName="test-curl-7411-140042772919040"
Jul 17 10:22:48.672794 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672791    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8852ac5-4d06-4b30-8c02-020fdbcd4b85" containerName="main"
Jul 17 10:22:48.672948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672797    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8852ac5-4d06-4b30-8c02-020fdbcd4b85" containerName="main"
Jul 17 10:22:48.672948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672877    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="98381cdd-fa62-4a2b-8f98-2c5d43217ba2" containerName="main"
Jul 17 10:22:48.672948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672886    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8852ac5-4d06-4b30-8c02-020fdbcd4b85" containerName="main"
Jul 17 10:22:48.672948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.672893    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f19ac6d-6cbb-412b-8204-2acb4135c802" containerName="test-curl-7411-140042772919040"
Jul 17 10:22:48.675981 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.675947    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:48.680571 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.680536    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:48.717096 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.717056    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjbv5\" (UniqueName: \"kubernetes.io/projected/7f19ac6d-6cbb-412b-8204-2acb4135c802-kube-api-access-jjbv5\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:48.817658 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.817616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5wsc\" (UniqueName: \"kubernetes.io/projected/0c182f98-a55e-4b0c-af41-4010d242093f-kube-api-access-j5wsc\") pod \"test-curl-7411-140042772919040\" (UID: \"0c182f98-a55e-4b0c-af41-4010d242093f\") " pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:48.918161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.918062    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5wsc\" (UniqueName: \"kubernetes.io/projected/0c182f98-a55e-4b0c-af41-4010d242093f-kube-api-access-j5wsc\") pod \"test-curl-7411-140042772919040\" (UID: \"0c182f98-a55e-4b0c-af41-4010d242093f\") " pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:48.926628 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.926585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5wsc\" (UniqueName: \"kubernetes.io/projected/0c182f98-a55e-4b0c-af41-4010d242093f-kube-api-access-j5wsc\") pod \"test-curl-7411-140042772919040\" (UID: \"0c182f98-a55e-4b0c-af41-4010d242093f\") " pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:48.955957 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.955923    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"]
Jul 17 10:22:48.959300 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.959275    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:22:48.961946 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.961920    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:22:48.962077 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.961924    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:22:48.967166 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.967129    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"]
Jul 17 10:22:48.986858 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:48.986800    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:49.119399 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.119362    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df47d942-bbc7-4203-9e05-c45e7febde31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wrz27\" (UID: \"df47d942-bbc7-4203-9e05-c45e7febde31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:22:49.127497 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.127417    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:49.130370 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:49.130330    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c182f98_a55e_4b0c_af41_4010d242093f.slice/crio-21e8dd42b647c51c65f1dd9ad8431bd273c224c951fa8d01d7e539efc11b6a0c WatchSource:0}: Error finding container 21e8dd42b647c51c65f1dd9ad8431bd273c224c951fa8d01d7e539efc11b6a0c: Status 404 returned error can't find the container with id 21e8dd42b647c51c65f1dd9ad8431bd273c224c951fa8d01d7e539efc11b6a0c
Jul 17 10:22:49.220233 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.220129    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df47d942-bbc7-4203-9e05-c45e7febde31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wrz27\" (UID: \"df47d942-bbc7-4203-9e05-c45e7febde31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:22:49.220940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.220911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df47d942-bbc7-4203-9e05-c45e7febde31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wrz27\" (UID: \"df47d942-bbc7-4203-9e05-c45e7febde31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:22:49.273372 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.273335    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:22:49.356487 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.356446    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:49.356653 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.356501    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31749fda7f939230a1dbda398c0e9ba2aed6acec22a6e9db058b8318bce01d8e"
Jul 17 10:22:49.357956 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.357920    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772919040" event={"ID":"0c182f98-a55e-4b0c-af41-4010d242093f","Type":"ContainerStarted","Data":"21e8dd42b647c51c65f1dd9ad8431bd273c224c951fa8d01d7e539efc11b6a0c"}
Jul 17 10:22:49.369376 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.369343    2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7411-140042772919040" oldPodUID="7f19ac6d-6cbb-412b-8204-2acb4135c802" podUID="0c182f98-a55e-4b0c-af41-4010d242093f"
Jul 17 10:22:49.412671 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:49.412622    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"]
Jul 17 10:22:49.416991 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:49.416963    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf47d942_bbc7_4203_9e05_c45e7febde31.slice/crio-eaed4a9d851db5563bbd592576d98ba09e89b91b80ff85dad5fabe0755f20f65 WatchSource:0}: Error finding container eaed4a9d851db5563bbd592576d98ba09e89b91b80ff85dad5fabe0755f20f65: Status 404 returned error can't find the container with id eaed4a9d851db5563bbd592576d98ba09e89b91b80ff85dad5fabe0755f20f65
Jul 17 10:22:50.365445 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.365341    2556 generic.go:356] "Generic (PLEG): container finished" podID="0c182f98-a55e-4b0c-af41-4010d242093f" containerID="b1f9b37a7b274576c49e811dfcc6afd8aec72b2dacd39b5e4c16aa3d916a027d" exitCode=0
Jul 17 10:22:50.365909 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.365436    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7411-140042772919040" event={"ID":"0c182f98-a55e-4b0c-af41-4010d242093f","Type":"ContainerDied","Data":"b1f9b37a7b274576c49e811dfcc6afd8aec72b2dacd39b5e4c16aa3d916a027d"}
Jul 17 10:22:50.366947 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.366919    2556 generic.go:356] "Generic (PLEG): container finished" podID="df47d942-bbc7-4203-9e05-c45e7febde31" containerID="50bba586322b8acd315816d8f2b67725576409726456d5a5802b2a7ce4576aa4" exitCode=6
Jul 17 10:22:50.367070 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.367006    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" event={"ID":"df47d942-bbc7-4203-9e05-c45e7febde31","Type":"ContainerDied","Data":"50bba586322b8acd315816d8f2b67725576409726456d5a5802b2a7ce4576aa4"}
Jul 17 10:22:50.367113 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.367082    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" event={"ID":"df47d942-bbc7-4203-9e05-c45e7febde31","Type":"ContainerStarted","Data":"eaed4a9d851db5563bbd592576d98ba09e89b91b80ff85dad5fabe0755f20f65"}
Jul 17 10:22:50.367174 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.367160    2556 scope.go:117] "RemoveContainer" containerID="50bba586322b8acd315816d8f2b67725576409726456d5a5802b2a7ce4576aa4"
Jul 17 10:22:50.495508 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:50.495478    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f19ac6d-6cbb-412b-8204-2acb4135c802" path="/var/lib/kubelet/pods/7f19ac6d-6cbb-412b-8204-2acb4135c802/volumes"
Jul 17 10:22:51.373283 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.373249    2556 generic.go:356] "Generic (PLEG): container finished" podID="df47d942-bbc7-4203-9e05-c45e7febde31" containerID="6d83578ae2985de2283e124d5682c6ca8146b009f4dcc37872e193d1cac12780" exitCode=6
Jul 17 10:22:51.373744 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.373344    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" event={"ID":"df47d942-bbc7-4203-9e05-c45e7febde31","Type":"ContainerDied","Data":"6d83578ae2985de2283e124d5682c6ca8146b009f4dcc37872e193d1cac12780"}
Jul 17 10:22:51.373744 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.373418    2556 scope.go:117] "RemoveContainer" containerID="50bba586322b8acd315816d8f2b67725576409726456d5a5802b2a7ce4576aa4"
Jul 17 10:22:51.373744 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.373514    2556 scope.go:117] "RemoveContainer" containerID="6d83578ae2985de2283e124d5682c6ca8146b009f4dcc37872e193d1cac12780"
Jul 17 10:22:51.373744 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:51.373705    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wrz27_odh-ai-gateway-infra(df47d942-bbc7-4203-9e05-c45e7febde31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" podUID="df47d942-bbc7-4203-9e05-c45e7febde31"
Jul 17 10:22:51.510672 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.510639    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:51.532991 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.532938    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7411-140042772919040_0c182f98-a55e-4b0c-af41-4010d242093f/test-curl-7411-140042772919040/0.log"
Jul 17 10:22:51.553803 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.553765    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:51.555210 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.555190    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7411-140042772919040"]
Jul 17 10:22:51.640213 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.640123    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5wsc\" (UniqueName: \"kubernetes.io/projected/0c182f98-a55e-4b0c-af41-4010d242093f-kube-api-access-j5wsc\") pod \"0c182f98-a55e-4b0c-af41-4010d242093f\" (UID: \"0c182f98-a55e-4b0c-af41-4010d242093f\") "
Jul 17 10:22:51.642504 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.642464    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c182f98-a55e-4b0c-af41-4010d242093f-kube-api-access-j5wsc" (OuterVolumeSpecName: "kube-api-access-j5wsc") pod "0c182f98-a55e-4b0c-af41-4010d242093f" (UID: "0c182f98-a55e-4b0c-af41-4010d242093f"). InnerVolumeSpecName "kube-api-access-j5wsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:22:51.741186 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.741140    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5wsc\" (UniqueName: \"kubernetes.io/projected/0c182f98-a55e-4b0c-af41-4010d242093f-kube-api-access-j5wsc\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:51.741722 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.741692    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"]
Jul 17 10:22:51.742152 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.742136    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c182f98-a55e-4b0c-af41-4010d242093f" containerName="test-curl-7411-140042772919040"
Jul 17 10:22:51.742224 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.742155    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c182f98-a55e-4b0c-af41-4010d242093f" containerName="test-curl-7411-140042772919040"
Jul 17 10:22:51.742272 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.742263    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c182f98-a55e-4b0c-af41-4010d242093f" containerName="test-curl-7411-140042772919040"
Jul 17 10:22:51.745512 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.745491    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:51.752734 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.752704    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"]
Jul 17 10:22:51.841959 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.841919    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2929085e-76b2-45ec-9523-be9eb28a5e04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw\" (UID: \"2929085e-76b2-45ec-9523-be9eb28a5e04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:51.942431 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.942344    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2929085e-76b2-45ec-9523-be9eb28a5e04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw\" (UID: \"2929085e-76b2-45ec-9523-be9eb28a5e04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:51.943038 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:51.943018    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2929085e-76b2-45ec-9523-be9eb28a5e04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw\" (UID: \"2929085e-76b2-45ec-9523-be9eb28a5e04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:52.058265 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.058223    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:52.198336 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.198309    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"]
Jul 17 10:22:52.200934 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:52.200898    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2929085e_76b2_45ec_9523_be9eb28a5e04.slice/crio-ca20a22c6ef158774c03559f5ac21733b7a3aaa95485f6989604315d4e343e76 WatchSource:0}: Error finding container ca20a22c6ef158774c03559f5ac21733b7a3aaa95485f6989604315d4e343e76: Status 404 returned error can't find the container with id ca20a22c6ef158774c03559f5ac21733b7a3aaa95485f6989604315d4e343e76
Jul 17 10:22:52.381233 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.381192    2556 scope.go:117] "RemoveContainer" containerID="6d83578ae2985de2283e124d5682c6ca8146b009f4dcc37872e193d1cac12780"
Jul 17 10:22:52.381661 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:52.381417    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wrz27_odh-ai-gateway-infra(df47d942-bbc7-4203-9e05-c45e7febde31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" podUID="df47d942-bbc7-4203-9e05-c45e7febde31"
Jul 17 10:22:52.382392 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.382360    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw" event={"ID":"2929085e-76b2-45ec-9523-be9eb28a5e04","Type":"ContainerStarted","Data":"ca20a22c6ef158774c03559f5ac21733b7a3aaa95485f6989604315d4e343e76"}
Jul 17 10:22:52.383859 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.383834    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:52.384013 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.383871    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21e8dd42b647c51c65f1dd9ad8431bd273c224c951fa8d01d7e539efc11b6a0c"
Jul 17 10:22:52.397071 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:22:52.397030    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7411-140042772919040\" is forbidden: User \"system:node:ip-10-0-141-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-244.ec2.internal' and this object" podUID="0c182f98-a55e-4b0c-af41-4010d242093f" pod="opendatahub/test-curl-7411-140042772919040"
Jul 17 10:22:52.495450 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.495414    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c182f98-a55e-4b0c-af41-4010d242093f" path="/var/lib/kubelet/pods/0c182f98-a55e-4b0c-af41-4010d242093f/volumes"
Jul 17 10:22:52.542698 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.542662    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:22:52.650886 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:52.650809    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:22:53.390071 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:53.390031    2556 generic.go:356] "Generic (PLEG): container finished" podID="2929085e-76b2-45ec-9523-be9eb28a5e04" containerID="e8fba431183a8232a94659d5bdd1c2a01b3542c83ac99c5c1d935cae593f0670" exitCode=0
Jul 17 10:22:53.390509 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:53.390125    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw" event={"ID":"2929085e-76b2-45ec-9523-be9eb28a5e04","Type":"ContainerDied","Data":"e8fba431183a8232a94659d5bdd1c2a01b3542c83ac99c5c1d935cae593f0670"}
Jul 17 10:22:54.536727 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:54.536687    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:54.667672 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:54.667580    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2929085e-76b2-45ec-9523-be9eb28a5e04-maas-api-service-ca\") pod \"2929085e-76b2-45ec-9523-be9eb28a5e04\" (UID: \"2929085e-76b2-45ec-9523-be9eb28a5e04\") "
Jul 17 10:22:54.667989 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:54.667965    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2929085e-76b2-45ec-9523-be9eb28a5e04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2929085e-76b2-45ec-9523-be9eb28a5e04" (UID: "2929085e-76b2-45ec-9523-be9eb28a5e04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:22:54.773510 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:54.769318    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2929085e-76b2-45ec-9523-be9eb28a5e04-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:22:55.400630 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:55.400596    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw"
Jul 17 10:22:55.400880 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:55.400636    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw" event={"ID":"2929085e-76b2-45ec-9523-be9eb28a5e04","Type":"ContainerDied","Data":"ca20a22c6ef158774c03559f5ac21733b7a3aaa95485f6989604315d4e343e76"}
Jul 17 10:22:55.400880 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:55.400714    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca20a22c6ef158774c03559f5ac21733b7a3aaa95485f6989604315d4e343e76"
Jul 17 10:22:58.066316 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.066280    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"]
Jul 17 10:22:58.066705 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.066587    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2929085e-76b2-45ec-9523-be9eb28a5e04" containerName="revoke-keys"
Jul 17 10:22:58.066705 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.066598    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2929085e-76b2-45ec-9523-be9eb28a5e04" containerName="revoke-keys"
Jul 17 10:22:58.066705 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.066663    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2929085e-76b2-45ec-9523-be9eb28a5e04" containerName="revoke-keys"
Jul 17 10:22:58.070772 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.070751    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:22:58.076061 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.076027    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"]
Jul 17 10:22:58.199788 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.199754    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d08a3ab0-823f-4532-a01c-6fc4da930016-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-ffprt\" (UID: \"d08a3ab0-823f-4532-a01c-6fc4da930016\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:22:58.300542 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.300496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d08a3ab0-823f-4532-a01c-6fc4da930016-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-ffprt\" (UID: \"d08a3ab0-823f-4532-a01c-6fc4da930016\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:22:58.301178 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.301155    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d08a3ab0-823f-4532-a01c-6fc4da930016-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-ffprt\" (UID: \"d08a3ab0-823f-4532-a01c-6fc4da930016\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:22:58.383785 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.383739    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:22:58.524964 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:58.524933    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"]
Jul 17 10:22:58.527337 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:22:58.527306    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08a3ab0_823f_4532_a01c_6fc4da930016.slice/crio-6b272b1bda33f56dcff4877c56f87941cd96ec95615ded2cc3860c4ef1b6b528 WatchSource:0}: Error finding container 6b272b1bda33f56dcff4877c56f87941cd96ec95615ded2cc3860c4ef1b6b528: Status 404 returned error can't find the container with id 6b272b1bda33f56dcff4877c56f87941cd96ec95615ded2cc3860c4ef1b6b528
Jul 17 10:22:59.421333 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:59.421271    2556 generic.go:356] "Generic (PLEG): container finished" podID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerID="d90f7cba7b054cfdc1430b7d35104635475ae8b846c62eb724985873c067900d" exitCode=6
Jul 17 10:22:59.421900 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:59.421356    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" event={"ID":"d08a3ab0-823f-4532-a01c-6fc4da930016","Type":"ContainerDied","Data":"d90f7cba7b054cfdc1430b7d35104635475ae8b846c62eb724985873c067900d"}
Jul 17 10:22:59.421900 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:59.421442    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" event={"ID":"d08a3ab0-823f-4532-a01c-6fc4da930016","Type":"ContainerStarted","Data":"6b272b1bda33f56dcff4877c56f87941cd96ec95615ded2cc3860c4ef1b6b528"}
Jul 17 10:22:59.421900 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:22:59.421676    2556 scope.go:117] "RemoveContainer" containerID="d90f7cba7b054cfdc1430b7d35104635475ae8b846c62eb724985873c067900d"
Jul 17 10:23:00.428302 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:00.428263    2556 generic.go:356] "Generic (PLEG): container finished" podID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerID="ab7ec1a407ea9a7f28ff3ebc0664737460dbe9779599cb908719f027db1b80f4" exitCode=6
Jul 17 10:23:00.428688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:00.428357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" event={"ID":"d08a3ab0-823f-4532-a01c-6fc4da930016","Type":"ContainerDied","Data":"ab7ec1a407ea9a7f28ff3ebc0664737460dbe9779599cb908719f027db1b80f4"}
Jul 17 10:23:00.428688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:00.428417    2556 scope.go:117] "RemoveContainer" containerID="d90f7cba7b054cfdc1430b7d35104635475ae8b846c62eb724985873c067900d"
Jul 17 10:23:00.428688 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:00.428565    2556 scope.go:117] "RemoveContainer" containerID="ab7ec1a407ea9a7f28ff3ebc0664737460dbe9779599cb908719f027db1b80f4"
Jul 17 10:23:00.428870 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:00.428767    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-ffprt_odh-ai-gateway-infra(d08a3ab0-823f-4532-a01c-6fc4da930016)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016"
Jul 17 10:23:01.434774 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:01.434741    2556 scope.go:117] "RemoveContainer" containerID="ab7ec1a407ea9a7f28ff3ebc0664737460dbe9779599cb908719f027db1b80f4"
Jul 17 10:23:01.435234 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:01.434969    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-ffprt_odh-ai-gateway-infra(d08a3ab0-823f-4532-a01c-6fc4da930016)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016"
Jul 17 10:23:02.313097 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:02.313041    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:03.489593 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:03.489563    2556 scope.go:117] "RemoveContainer" containerID="6d83578ae2985de2283e124d5682c6ca8146b009f4dcc37872e193d1cac12780"
Jul 17 10:23:04.452246 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:04.452164    2556 generic.go:356] "Generic (PLEG): container finished" podID="df47d942-bbc7-4203-9e05-c45e7febde31" containerID="65158c2d60596cf705ddb97480e8a0941299890af8e33a93197f57ec7924b1d9" exitCode=6
Jul 17 10:23:04.452246 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:04.452233    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" event={"ID":"df47d942-bbc7-4203-9e05-c45e7febde31","Type":"ContainerDied","Data":"65158c2d60596cf705ddb97480e8a0941299890af8e33a93197f57ec7924b1d9"}
Jul 17 10:23:04.452425 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:04.452294    2556 scope.go:117] "RemoveContainer" containerID="6d83578ae2985de2283e124d5682c6ca8146b009f4dcc37872e193d1cac12780"
Jul 17 10:23:04.452645 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:04.452626    2556 scope.go:117] "RemoveContainer" containerID="65158c2d60596cf705ddb97480e8a0941299890af8e33a93197f57ec7924b1d9"
Jul 17 10:23:04.452856 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:04.452810    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wrz27_odh-ai-gateway-infra(df47d942-bbc7-4203-9e05-c45e7febde31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" podUID="df47d942-bbc7-4203-9e05-c45e7febde31"
Jul 17 10:23:05.485435 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:05.485395    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"]
Jul 17 10:23:05.625198 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:05.625165    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:23:05.657712 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:05.657677    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df47d942-bbc7-4203-9e05-c45e7febde31-maas-api-service-ca\") pod \"df47d942-bbc7-4203-9e05-c45e7febde31\" (UID: \"df47d942-bbc7-4203-9e05-c45e7febde31\") "
Jul 17 10:23:05.658034 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:05.658013    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df47d942-bbc7-4203-9e05-c45e7febde31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df47d942-bbc7-4203-9e05-c45e7febde31" (UID: "df47d942-bbc7-4203-9e05-c45e7febde31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:05.758384 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:05.758286    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df47d942-bbc7-4203-9e05-c45e7febde31-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:23:06.173992 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.173952    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:23:06.331343 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331307    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"]
Jul 17 10:23:06.331626 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331613    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:06.331677 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331627    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:06.331677 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331640    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:06.331677 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331646    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:06.331764 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331698    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:06.331764 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.331707    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:06.335246 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.335226    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:06.341866 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.341811    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"]
Jul 17 10:23:06.361297 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.361263    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbedb5eb-852e-4927-b073-e108a18eeafd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-qmzhg\" (UID: \"bbedb5eb-852e-4927-b073-e108a18eeafd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:06.462249 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.462146    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbedb5eb-852e-4927-b073-e108a18eeafd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-qmzhg\" (UID: \"bbedb5eb-852e-4927-b073-e108a18eeafd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:06.462847 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.462796    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbedb5eb-852e-4927-b073-e108a18eeafd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-qmzhg\" (UID: \"bbedb5eb-852e-4927-b073-e108a18eeafd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:06.465579 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.465552    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"
Jul 17 10:23:06.465579 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.465558    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27" event={"ID":"df47d942-bbc7-4203-9e05-c45e7febde31","Type":"ContainerDied","Data":"eaed4a9d851db5563bbd592576d98ba09e89b91b80ff85dad5fabe0755f20f65"}
Jul 17 10:23:06.465791 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.465636    2556 scope.go:117] "RemoveContainer" containerID="65158c2d60596cf705ddb97480e8a0941299890af8e33a93197f57ec7924b1d9"
Jul 17 10:23:06.486510 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.486474    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"]
Jul 17 10:23:06.496245 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.496213    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wrz27"]
Jul 17 10:23:06.648351 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.648298    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:06.793748 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.793711    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"]
Jul 17 10:23:06.796864 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:23:06.796802    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbedb5eb_852e_4927_b073_e108a18eeafd.slice/crio-535dc22ab57503aaa62ae4ee6ac24c38a0bc8be68b5289a518f3f7465d00f002 WatchSource:0}: Error finding container 535dc22ab57503aaa62ae4ee6ac24c38a0bc8be68b5289a518f3f7465d00f002: Status 404 returned error can't find the container with id 535dc22ab57503aaa62ae4ee6ac24c38a0bc8be68b5289a518f3f7465d00f002
Jul 17 10:23:06.869723 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:06.869684    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:07.472777 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:07.472735    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" event={"ID":"bbedb5eb-852e-4927-b073-e108a18eeafd","Type":"ContainerStarted","Data":"3d92f6ad277d5889bdc5ea49e98478642f99efba8015850064967c8b8126cf24"}
Jul 17 10:23:07.472977 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:07.472792    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" event={"ID":"bbedb5eb-852e-4927-b073-e108a18eeafd","Type":"ContainerStarted","Data":"535dc22ab57503aaa62ae4ee6ac24c38a0bc8be68b5289a518f3f7465d00f002"}
Jul 17 10:23:07.489306 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:07.489247    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" podStartSLOduration=1.4892233049999999 podStartE2EDuration="1.489223305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:23:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:23:07.48670898 +0000 UTC m=+3371.496832905" watchObservedRunningTime="2026-07-17 10:23:07.489223305 +0000 UTC m=+3371.499347229"
Jul 17 10:23:08.479945 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:08.479908    2556 generic.go:356] "Generic (PLEG): container finished" podID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerID="3d92f6ad277d5889bdc5ea49e98478642f99efba8015850064967c8b8126cf24" exitCode=6
Jul 17 10:23:08.480117 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:08.479998    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" event={"ID":"bbedb5eb-852e-4927-b073-e108a18eeafd","Type":"ContainerDied","Data":"3d92f6ad277d5889bdc5ea49e98478642f99efba8015850064967c8b8126cf24"}
Jul 17 10:23:08.480211 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:08.480194    2556 scope.go:117] "RemoveContainer" containerID="3d92f6ad277d5889bdc5ea49e98478642f99efba8015850064967c8b8126cf24"
Jul 17 10:23:08.495489 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:08.495459    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" path="/var/lib/kubelet/pods/df47d942-bbc7-4203-9e05-c45e7febde31/volumes"
Jul 17 10:23:09.486922 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:09.486886    2556 generic.go:356] "Generic (PLEG): container finished" podID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerID="d2db34493c0e629368351a7b50d2a1e1ac7267009b53db37ba1684d766606f6d" exitCode=6
Jul 17 10:23:09.487205 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:09.486942    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" event={"ID":"bbedb5eb-852e-4927-b073-e108a18eeafd","Type":"ContainerDied","Data":"d2db34493c0e629368351a7b50d2a1e1ac7267009b53db37ba1684d766606f6d"}
Jul 17 10:23:09.487205 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:09.486998    2556 scope.go:117] "RemoveContainer" containerID="3d92f6ad277d5889bdc5ea49e98478642f99efba8015850064967c8b8126cf24"
Jul 17 10:23:09.487205 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:09.487180    2556 scope.go:117] "RemoveContainer" containerID="d2db34493c0e629368351a7b50d2a1e1ac7267009b53db37ba1684d766606f6d"
Jul 17 10:23:09.487424 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:09.487398    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-qmzhg_odh-ai-gateway-infra(bbedb5eb-852e-4927-b073-e108a18eeafd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd"
Jul 17 10:23:10.493940 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:10.493911    2556 scope.go:117] "RemoveContainer" containerID="d2db34493c0e629368351a7b50d2a1e1ac7267009b53db37ba1684d766606f6d"
Jul 17 10:23:10.494323 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:10.494109    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-qmzhg_odh-ai-gateway-infra(bbedb5eb-852e-4927-b073-e108a18eeafd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd"
Jul 17 10:23:12.949733 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:12.949672    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:13.081024 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:13.080973    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:13.490000 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:13.489967    2556 scope.go:117] "RemoveContainer" containerID="ab7ec1a407ea9a7f28ff3ebc0664737460dbe9779599cb908719f027db1b80f4"
Jul 17 10:23:14.520210 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:14.520125    2556 generic.go:356] "Generic (PLEG): container finished" podID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerID="b21b77ce54b0c576e53a1ffd975a6dff139af2d308e6b8dc9cff65c2906a42d2" exitCode=6
Jul 17 10:23:14.520210 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:14.520197    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" event={"ID":"d08a3ab0-823f-4532-a01c-6fc4da930016","Type":"ContainerDied","Data":"b21b77ce54b0c576e53a1ffd975a6dff139af2d308e6b8dc9cff65c2906a42d2"}
Jul 17 10:23:14.520677 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:14.520261    2556 scope.go:117] "RemoveContainer" containerID="ab7ec1a407ea9a7f28ff3ebc0664737460dbe9779599cb908719f027db1b80f4"
Jul 17 10:23:14.520677 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:14.520535    2556 scope.go:117] "RemoveContainer" containerID="b21b77ce54b0c576e53a1ffd975a6dff139af2d308e6b8dc9cff65c2906a42d2"
Jul 17 10:23:14.520787 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:14.520730    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-ffprt_odh-ai-gateway-infra(d08a3ab0-823f-4532-a01c-6fc4da930016)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016"
Jul 17 10:23:15.545788 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:15.545737    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"]
Jul 17 10:23:15.689036 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:15.689003    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:23:15.728735 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:15.728702    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d08a3ab0-823f-4532-a01c-6fc4da930016-maas-api-service-ca\") pod \"d08a3ab0-823f-4532-a01c-6fc4da930016\" (UID: \"d08a3ab0-823f-4532-a01c-6fc4da930016\") "
Jul 17 10:23:15.729122 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:15.729100    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d08a3ab0-823f-4532-a01c-6fc4da930016-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d08a3ab0-823f-4532-a01c-6fc4da930016" (UID: "d08a3ab0-823f-4532-a01c-6fc4da930016"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:15.829237 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:15.829146    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d08a3ab0-823f-4532-a01c-6fc4da930016-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:23:16.533731 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:16.533693    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt" event={"ID":"d08a3ab0-823f-4532-a01c-6fc4da930016","Type":"ContainerDied","Data":"6b272b1bda33f56dcff4877c56f87941cd96ec95615ded2cc3860c4ef1b6b528"}
Jul 17 10:23:16.533731 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:16.533725    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"
Jul 17 10:23:16.533990 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:16.533780    2556 scope.go:117] "RemoveContainer" containerID="b21b77ce54b0c576e53a1ffd975a6dff139af2d308e6b8dc9cff65c2906a42d2"
Jul 17 10:23:16.549555 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:16.549522    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"]
Jul 17 10:23:16.556862 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:16.556812    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ffprt"]
Jul 17 10:23:17.596933 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.596873    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:17.846616 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.846578    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:17.963456 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963420    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"]
Jul 17 10:23:17.963736 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963724    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.963781 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963738    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.963781 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963752    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:17.963781 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963758    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:17.963781 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963764    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.963781 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963770    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.963781 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963779    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.964030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963785    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.964030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963851    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.964030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963860    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.964030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963867    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" containerName="revoke-keys"
Jul 17 10:23:17.964030 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.963872    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="df47d942-bbc7-4203-9e05-c45e7febde31" containerName="revoke-keys"
Jul 17 10:23:17.968146 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.968126    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:17.974897 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:17.974631    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"]
Jul 17 10:23:18.045302 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.045262    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd506bc-2eae-4ce8-b7cf-cdf032e90dea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx\" (UID: \"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:18.146633 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.146587    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd506bc-2eae-4ce8-b7cf-cdf032e90dea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx\" (UID: \"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:18.147292 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.147264    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd506bc-2eae-4ce8-b7cf-cdf032e90dea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx\" (UID: \"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:18.279354 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.279258    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:18.421010 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.420738    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"]
Jul 17 10:23:18.423659 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:23:18.423629    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd506bc_2eae_4ce8_b7cf_cdf032e90dea.slice/crio-7628687559d911827a162b1a29a5ac3254cba984f55c852063b44ff6da803db9 WatchSource:0}: Error finding container 7628687559d911827a162b1a29a5ac3254cba984f55c852063b44ff6da803db9: Status 404 returned error can't find the container with id 7628687559d911827a162b1a29a5ac3254cba984f55c852063b44ff6da803db9
Jul 17 10:23:18.495103 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.495067    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08a3ab0-823f-4532-a01c-6fc4da930016" path="/var/lib/kubelet/pods/d08a3ab0-823f-4532-a01c-6fc4da930016/volumes"
Jul 17 10:23:18.517182 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.517140    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:18.546315 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.546221    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx" event={"ID":"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea","Type":"ContainerStarted","Data":"7628687559d911827a162b1a29a5ac3254cba984f55c852063b44ff6da803db9"}
Jul 17 10:23:18.618250 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:18.618196    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:19.553146 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:19.553101    2556 generic.go:356] "Generic (PLEG): container finished" podID="cdd506bc-2eae-4ce8-b7cf-cdf032e90dea" containerID="8a06b97cd7e285d68b95b9a51d83a8036769bf10d4164121f5c42c03c1c1838f" exitCode=0
Jul 17 10:23:19.553329 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:19.553199    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx" event={"ID":"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea","Type":"ContainerDied","Data":"8a06b97cd7e285d68b95b9a51d83a8036769bf10d4164121f5c42c03c1c1838f"}
Jul 17 10:23:20.705503 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:20.705468    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:20.764707 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:20.764675    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd506bc-2eae-4ce8-b7cf-cdf032e90dea-maas-api-service-ca\") pod \"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea\" (UID: \"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea\") "
Jul 17 10:23:20.765092 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:20.765068    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd506bc-2eae-4ce8-b7cf-cdf032e90dea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cdd506bc-2eae-4ce8-b7cf-cdf032e90dea" (UID: "cdd506bc-2eae-4ce8-b7cf-cdf032e90dea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:20.865969 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:20.865793    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd506bc-2eae-4ce8-b7cf-cdf032e90dea-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:23:21.490513 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:21.490479    2556 scope.go:117] "RemoveContainer" containerID="d2db34493c0e629368351a7b50d2a1e1ac7267009b53db37ba1684d766606f6d"
Jul 17 10:23:21.564003 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:21.563973    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx"
Jul 17 10:23:21.564003 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:21.563986    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx" event={"ID":"cdd506bc-2eae-4ce8-b7cf-cdf032e90dea","Type":"ContainerDied","Data":"7628687559d911827a162b1a29a5ac3254cba984f55c852063b44ff6da803db9"}
Jul 17 10:23:21.564218 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:21.564035    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7628687559d911827a162b1a29a5ac3254cba984f55c852063b44ff6da803db9"
Jul 17 10:23:22.570016 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:22.569978    2556 generic.go:356] "Generic (PLEG): container finished" podID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerID="7e116546a62cd14f922cfc590dd7b9fe67386181ce8b1e9747ce3ed3e5fb1323" exitCode=6
Jul 17 10:23:22.570466 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:22.570046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" event={"ID":"bbedb5eb-852e-4927-b073-e108a18eeafd","Type":"ContainerDied","Data":"7e116546a62cd14f922cfc590dd7b9fe67386181ce8b1e9747ce3ed3e5fb1323"}
Jul 17 10:23:22.570466 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:22.570111    2556 scope.go:117] "RemoveContainer" containerID="d2db34493c0e629368351a7b50d2a1e1ac7267009b53db37ba1684d766606f6d"
Jul 17 10:23:22.570466 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:22.570424    2556 scope.go:117] "RemoveContainer" containerID="7e116546a62cd14f922cfc590dd7b9fe67386181ce8b1e9747ce3ed3e5fb1323"
Jul 17 10:23:22.570681 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:22.570653    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-qmzhg_odh-ai-gateway-infra(bbedb5eb-852e-4927-b073-e108a18eeafd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd"
Jul 17 10:23:23.597048 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:23.597014    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"]
Jul 17 10:23:23.732936 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:23.732906    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:23.788198 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:23.788160    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbedb5eb-852e-4927-b073-e108a18eeafd-maas-api-service-ca\") pod \"bbedb5eb-852e-4927-b073-e108a18eeafd\" (UID: \"bbedb5eb-852e-4927-b073-e108a18eeafd\") "
Jul 17 10:23:23.788613 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:23.788591    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbedb5eb-852e-4927-b073-e108a18eeafd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbedb5eb-852e-4927-b073-e108a18eeafd" (UID: "bbedb5eb-852e-4927-b073-e108a18eeafd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:23.888772 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:23.888732    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbedb5eb-852e-4927-b073-e108a18eeafd-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:23:24.583728 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:24.583693    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg" event={"ID":"bbedb5eb-852e-4927-b073-e108a18eeafd","Type":"ContainerDied","Data":"535dc22ab57503aaa62ae4ee6ac24c38a0bc8be68b5289a518f3f7465d00f002"}
Jul 17 10:23:24.583728 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:24.583730    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"
Jul 17 10:23:24.583991 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:24.583757    2556 scope.go:117] "RemoveContainer" containerID="7e116546a62cd14f922cfc590dd7b9fe67386181ce8b1e9747ce3ed3e5fb1323"
Jul 17 10:23:24.598966 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:24.598932    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"]
Jul 17 10:23:24.603069 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:24.603039    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-qmzhg"]
Jul 17 10:23:26.496626 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.496589    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" path="/var/lib/kubelet/pods/bbedb5eb-852e-4927-b073-e108a18eeafd/volumes"
Jul 17 10:23:26.866597 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866506    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"]
Jul 17 10:23:26.866805 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866794    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866807    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866838    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd506bc-2eae-4ce8-b7cf-cdf032e90dea" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866847    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd506bc-2eae-4ce8-b7cf-cdf032e90dea" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866859    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866865    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866875    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.866896 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866880    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.867148 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866929    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.867148 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866941    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.867148 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866946    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd506bc-2eae-4ce8-b7cf-cdf032e90dea" containerName="revoke-keys"
Jul 17 10:23:26.867148 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.866953    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbedb5eb-852e-4927-b073-e108a18eeafd" containerName="revoke-keys"
Jul 17 10:23:26.872057 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.871966    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:26.875586 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.875560    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:23:26.875808 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.875568    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 10:23:26.876968 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.876935    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"]
Jul 17 10:23:26.912438 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:26.912392    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg\" (UID: \"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:27.013089 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.013048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg\" (UID: \"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:27.013703 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.013680    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg\" (UID: \"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:27.186760 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.186717    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:27.329233 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.329203    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"]
Jul 17 10:23:27.331882 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:23:27.331848    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c27ec7_453a_47a6_ae8c_abdfbc4b8dec.slice/crio-fe04c7aa484bed8d81969d7873fcffe5431e961f6a7349bffc3d371b1910ddd3 WatchSource:0}: Error finding container fe04c7aa484bed8d81969d7873fcffe5431e961f6a7349bffc3d371b1910ddd3: Status 404 returned error can't find the container with id fe04c7aa484bed8d81969d7873fcffe5431e961f6a7349bffc3d371b1910ddd3
Jul 17 10:23:27.601333 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.601293    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" event={"ID":"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec","Type":"ContainerStarted","Data":"90da35f02f15a48033aef625c881a376258427bace8fec989e4b4777cfd87a6e"}
Jul 17 10:23:27.601755 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.601362    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" event={"ID":"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec","Type":"ContainerStarted","Data":"fe04c7aa484bed8d81969d7873fcffe5431e961f6a7349bffc3d371b1910ddd3"}
Jul 17 10:23:27.617606 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:27.617550    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" podStartSLOduration=1.617535476 podStartE2EDuration="1.617535476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:23:27.61577803 +0000 UTC m=+3391.625901955" watchObservedRunningTime="2026-07-17 10:23:27.617535476 +0000 UTC m=+3391.627659399"
Jul 17 10:23:28.437117 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:28.437075    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:28.607383 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:28.607349    2556 generic.go:356] "Generic (PLEG): container finished" podID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerID="90da35f02f15a48033aef625c881a376258427bace8fec989e4b4777cfd87a6e" exitCode=6
Jul 17 10:23:28.607789 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:28.607443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" event={"ID":"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec","Type":"ContainerDied","Data":"90da35f02f15a48033aef625c881a376258427bace8fec989e4b4777cfd87a6e"}
Jul 17 10:23:28.607789 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:28.607576    2556 scope.go:117] "RemoveContainer" containerID="90da35f02f15a48033aef625c881a376258427bace8fec989e4b4777cfd87a6e"
Jul 17 10:23:29.613740 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:29.613642    2556 generic.go:356] "Generic (PLEG): container finished" podID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerID="dfebea1091f7f25c8077ea59ea0d0c3eb7151a1c5e6235241982eb872b936542" exitCode=6
Jul 17 10:23:29.613740 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:29.613722    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" event={"ID":"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec","Type":"ContainerDied","Data":"dfebea1091f7f25c8077ea59ea0d0c3eb7151a1c5e6235241982eb872b936542"}
Jul 17 10:23:29.614288 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:29.613790    2556 scope.go:117] "RemoveContainer" containerID="90da35f02f15a48033aef625c881a376258427bace8fec989e4b4777cfd87a6e"
Jul 17 10:23:29.614288 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:29.613975    2556 scope.go:117] "RemoveContainer" containerID="dfebea1091f7f25c8077ea59ea0d0c3eb7151a1c5e6235241982eb872b936542"
Jul 17 10:23:29.614288 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:29.614188    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg_odh-ai-gateway-infra(a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec"
Jul 17 10:23:30.620881 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:30.620850    2556 scope.go:117] "RemoveContainer" containerID="dfebea1091f7f25c8077ea59ea0d0c3eb7151a1c5e6235241982eb872b936542"
Jul 17 10:23:30.621257 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:30.621040    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg_odh-ai-gateway-infra(a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec"
Jul 17 10:23:31.658976 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:31.658940    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bcxhk"]
Jul 17 10:23:33.065333 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:33.065290    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:38.997277 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:38.997242    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"]
Jul 17 10:23:39.000452 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.000429    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:39.010855 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.009660    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"]
Jul 17 10:23:39.066206 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.066168    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:39.110637 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.110595    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be78fa2c-49f1-4eb7-a94f-189c746ca556-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8\" (UID: \"be78fa2c-49f1-4eb7-a94f-189c746ca556\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:39.167637 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.167597    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:39.211901 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.211860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be78fa2c-49f1-4eb7-a94f-189c746ca556-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8\" (UID: \"be78fa2c-49f1-4eb7-a94f-189c746ca556\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:39.212482 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.212450    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be78fa2c-49f1-4eb7-a94f-189c746ca556-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8\" (UID: \"be78fa2c-49f1-4eb7-a94f-189c746ca556\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:39.316053 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.315972    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:39.450442 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.450415    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"]
Jul 17 10:23:39.452904 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:23:39.452877    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe78fa2c_49f1_4eb7_a94f_189c746ca556.slice/crio-ae3a6525d06c4206b43e250f99ca30697ff8da399051006f995b2cf968c9c1ac WatchSource:0}: Error finding container ae3a6525d06c4206b43e250f99ca30697ff8da399051006f995b2cf968c9c1ac: Status 404 returned error can't find the container with id ae3a6525d06c4206b43e250f99ca30697ff8da399051006f995b2cf968c9c1ac
Jul 17 10:23:39.665443 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:39.665405    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" event={"ID":"be78fa2c-49f1-4eb7-a94f-189c746ca556","Type":"ContainerStarted","Data":"ae3a6525d06c4206b43e250f99ca30697ff8da399051006f995b2cf968c9c1ac"}
Jul 17 10:23:40.671728 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:40.671686    2556 generic.go:356] "Generic (PLEG): container finished" podID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerID="b45211510088f619e7db2b38f8dd61ab462066acbb8e7da6eb4814599c3b6431" exitCode=6
Jul 17 10:23:40.672230 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:40.671777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" event={"ID":"be78fa2c-49f1-4eb7-a94f-189c746ca556","Type":"ContainerDied","Data":"b45211510088f619e7db2b38f8dd61ab462066acbb8e7da6eb4814599c3b6431"}
Jul 17 10:23:40.672230 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:40.671971    2556 scope.go:117] "RemoveContainer" containerID="b45211510088f619e7db2b38f8dd61ab462066acbb8e7da6eb4814599c3b6431"
Jul 17 10:23:41.677918 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:41.677885    2556 generic.go:356] "Generic (PLEG): container finished" podID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerID="d9fd400efdc8bd87a7c19bab024ee61e40d0e789e04a9b718441e9d4be2713c8" exitCode=6
Jul 17 10:23:41.678336 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:41.677971    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" event={"ID":"be78fa2c-49f1-4eb7-a94f-189c746ca556","Type":"ContainerDied","Data":"d9fd400efdc8bd87a7c19bab024ee61e40d0e789e04a9b718441e9d4be2713c8"}
Jul 17 10:23:41.678336 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:41.678027    2556 scope.go:117] "RemoveContainer" containerID="b45211510088f619e7db2b38f8dd61ab462066acbb8e7da6eb4814599c3b6431"
Jul 17 10:23:41.678336 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:41.678209    2556 scope.go:117] "RemoveContainer" containerID="d9fd400efdc8bd87a7c19bab024ee61e40d0e789e04a9b718441e9d4be2713c8"
Jul 17 10:23:41.678475 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:41.678440    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8_odh-ai-gateway-infra(be78fa2c-49f1-4eb7-a94f-189c746ca556)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556"
Jul 17 10:23:42.684999 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:42.684970    2556 scope.go:117] "RemoveContainer" containerID="d9fd400efdc8bd87a7c19bab024ee61e40d0e789e04a9b718441e9d4be2713c8"
Jul 17 10:23:42.685365 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:42.685136    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8_odh-ai-gateway-infra(be78fa2c-49f1-4eb7-a94f-189c746ca556)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556"
Jul 17 10:23:43.798231 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:43.798194    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:44.055107 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:44.055011    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:44.509047 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:44.509012    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b98754ddd-wkw2s_459e981a-6020-4de5-8706-caf932594fa2/authorino/0.log"
Jul 17 10:23:45.490156 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:45.490117    2556 scope.go:117] "RemoveContainer" containerID="dfebea1091f7f25c8077ea59ea0d0c3eb7151a1c5e6235241982eb872b936542"
Jul 17 10:23:46.706367 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:46.706287    2556 generic.go:356] "Generic (PLEG): container finished" podID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerID="280a07e510aeb8ad011fac538941a88bafe3f1bd1a0712d5e4a4c9bc8642180d" exitCode=6
Jul 17 10:23:46.706367 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:46.706344    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" event={"ID":"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec","Type":"ContainerDied","Data":"280a07e510aeb8ad011fac538941a88bafe3f1bd1a0712d5e4a4c9bc8642180d"}
Jul 17 10:23:46.706871 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:46.706393    2556 scope.go:117] "RemoveContainer" containerID="dfebea1091f7f25c8077ea59ea0d0c3eb7151a1c5e6235241982eb872b936542"
Jul 17 10:23:46.706871 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:46.706722    2556 scope.go:117] "RemoveContainer" containerID="280a07e510aeb8ad011fac538941a88bafe3f1bd1a0712d5e4a4c9bc8642180d"
Jul 17 10:23:46.706983 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:46.706956    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg_odh-ai-gateway-infra(a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec"
Jul 17 10:23:47.730638 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:47.730602    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"]
Jul 17 10:23:47.866980 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:47.866954    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:47.985028 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:47.984935    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec-maas-api-service-ca\") pod \"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec\" (UID: \"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec\") "
Jul 17 10:23:47.985318 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:47.985294    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" (UID: "a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:48.085809 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.085771    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:23:48.117554 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.117514    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"]
Jul 17 10:23:48.117899 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.117882    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:23:48.118001 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.117900    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:23:48.118001 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.117916    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:23:48.118001 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.117923    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:23:48.118149 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.118028    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:23:48.118149 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.118044    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:23:48.120953 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.120930    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:23:48.129814 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.129788    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"]
Jul 17 10:23:48.186765 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.186720    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dae72d0-6a02-42a0-913a-2d71d19f1994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9\" (UID: \"5dae72d0-6a02-42a0-913a-2d71d19f1994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:23:48.288223 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.288139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dae72d0-6a02-42a0-913a-2d71d19f1994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9\" (UID: \"5dae72d0-6a02-42a0-913a-2d71d19f1994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:23:48.288766 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.288745    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dae72d0-6a02-42a0-913a-2d71d19f1994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9\" (UID: \"5dae72d0-6a02-42a0-913a-2d71d19f1994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:23:48.432264 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.432225    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:23:48.568449 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.568422    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"]
Jul 17 10:23:48.571021 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:23:48.570985    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dae72d0_6a02_42a0_913a_2d71d19f1994.slice/crio-2dd4451cc8ae687aa1193e6f5a1b6fbca117c1f2d34547e42581063eb16e70a5 WatchSource:0}: Error finding container 2dd4451cc8ae687aa1193e6f5a1b6fbca117c1f2d34547e42581063eb16e70a5: Status 404 returned error can't find the container with id 2dd4451cc8ae687aa1193e6f5a1b6fbca117c1f2d34547e42581063eb16e70a5
Jul 17 10:23:48.719093 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.718989    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"
Jul 17 10:23:48.719093 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.719029    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg" event={"ID":"a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec","Type":"ContainerDied","Data":"fe04c7aa484bed8d81969d7873fcffe5431e961f6a7349bffc3d371b1910ddd3"}
Jul 17 10:23:48.736558 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.719091    2556 scope.go:117] "RemoveContainer" containerID="280a07e510aeb8ad011fac538941a88bafe3f1bd1a0712d5e4a4c9bc8642180d"
Jul 17 10:23:48.736558 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.720394    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" event={"ID":"5dae72d0-6a02-42a0-913a-2d71d19f1994","Type":"ContainerStarted","Data":"2dd4451cc8ae687aa1193e6f5a1b6fbca117c1f2d34547e42581063eb16e70a5"}
Jul 17 10:23:48.736558 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.734422    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"]
Jul 17 10:23:48.739337 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:48.739310    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-g2rzg"]
Jul 17 10:23:49.728026 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:49.727991    2556 generic.go:356] "Generic (PLEG): container finished" podID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerID="e3aeb27fd84b1a63fa87208fa264848c331c74e9fe2155a7e2d0b21db74ba493" exitCode=6
Jul 17 10:23:49.728248 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:49.728041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" event={"ID":"5dae72d0-6a02-42a0-913a-2d71d19f1994","Type":"ContainerDied","Data":"e3aeb27fd84b1a63fa87208fa264848c331c74e9fe2155a7e2d0b21db74ba493"}
Jul 17 10:23:49.728326 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:49.728291    2556 scope.go:117] "RemoveContainer" containerID="e3aeb27fd84b1a63fa87208fa264848c331c74e9fe2155a7e2d0b21db74ba493"
Jul 17 10:23:50.495371 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.495330    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" path="/var/lib/kubelet/pods/a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec/volumes"
Jul 17 10:23:50.734811 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.734773    2556 generic.go:356] "Generic (PLEG): container finished" podID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerID="68eeadec525cd9ca0b46dac2e4dbe4b9e5c5eacf86abfc2304b5c55e7a245e3c" exitCode=6
Jul 17 10:23:50.735019 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.734870    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" event={"ID":"5dae72d0-6a02-42a0-913a-2d71d19f1994","Type":"ContainerDied","Data":"68eeadec525cd9ca0b46dac2e4dbe4b9e5c5eacf86abfc2304b5c55e7a245e3c"}
Jul 17 10:23:50.735019 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.734939    2556 scope.go:117] "RemoveContainer" containerID="e3aeb27fd84b1a63fa87208fa264848c331c74e9fe2155a7e2d0b21db74ba493"
Jul 17 10:23:50.735168 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.735153    2556 scope.go:117] "RemoveContainer" containerID="68eeadec525cd9ca0b46dac2e4dbe4b9e5c5eacf86abfc2304b5c55e7a245e3c"
Jul 17 10:23:50.735385 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:50.735367    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9_odh-ai-gateway-infra(5dae72d0-6a02-42a0-913a-2d71d19f1994)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994"
Jul 17 10:23:50.896696 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.896669    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-2q7sh_7fd30b3c-b529-4a08-a003-a63d1f4a002a/copy-manifests/0.log"
Jul 17 10:23:50.904659 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:50.904634    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-2q7sh_7fd30b3c-b529-4a08-a003-a63d1f4a002a/manager/0.log"
Jul 17 10:23:51.741801 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:51.741768    2556 scope.go:117] "RemoveContainer" containerID="68eeadec525cd9ca0b46dac2e4dbe4b9e5c5eacf86abfc2304b5c55e7a245e3c"
Jul 17 10:23:51.742287 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:51.742029    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9_odh-ai-gateway-infra(5dae72d0-6a02-42a0-913a-2d71d19f1994)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994"
Jul 17 10:23:51.963776 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:51.963744    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738055-zm7nc_979c9f59-4ace-4a90-ba49-bc17271f0994/cleanup/0.log"
Jul 17 10:23:52.080074 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:52.079993    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8_be78fa2c-49f1-4eb7-a94f-189c746ca556/revoke-keys/1.log"
Jul 17 10:23:52.198926 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:52.198895    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-d4c7ee-pgpqd_2e9e9264-4b59-4fc0-ba10-c96f05ef2494/revoke-keys/0.log"
Jul 17 10:23:52.317123 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:52.317089    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-1baf67-2hx92_7252eb94-3f1c-4373-ba34-645d4ce677fd/revoke-keys/0.log"
Jul 17 10:23:52.686685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:52.686658    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9_5dae72d0-6a02-42a0-913a-2d71d19f1994/revoke-keys/1.log"
Jul 17 10:23:53.055363 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:53.055283    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-ed096b-8qgnw_2929085e-76b2-45ec-9523-be9eb28a5e04/revoke-keys/0.log"
Jul 17 10:23:53.170642 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:53.170615    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-af80d9-dsvpx_cdd506bc-2eae-4ce8-b7cf-cdf032e90dea/revoke-keys/0.log"
Jul 17 10:23:54.402795 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:54.402764    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:23:54.640582 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:54.640546    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b98754ddd-wkw2s_459e981a-6020-4de5-8706-caf932594fa2/authorino/0.log"
Jul 17 10:23:54.996005 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:54.995975    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mvqhv_e23b0300-48e8-4515-acc2-79019874acea/kuadrant-console-plugin/0.log"
Jul 17 10:23:55.376512 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:55.376462    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bcxhk_2be2706b-989b-40d8-babb-237b01f5c399/limitador/0.log"
Jul 17 10:23:55.489970 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:55.489935    2556 scope.go:117] "RemoveContainer" containerID="d9fd400efdc8bd87a7c19bab024ee61e40d0e789e04a9b718441e9d4be2713c8"
Jul 17 10:23:56.159967 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:56.159933    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-jhjq6_b5fb7163-04a7-4c5a-b251-e0c4bd52f927/kube-auth-proxy/0.log"
Jul 17 10:23:56.767814 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:56.767778    2556 generic.go:356] "Generic (PLEG): container finished" podID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerID="8d559bb25acd6e418365c7307fa5788e3524816cf14acb53956d46cc1b3050e6" exitCode=6
Jul 17 10:23:56.768268 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:56.767846    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" event={"ID":"be78fa2c-49f1-4eb7-a94f-189c746ca556","Type":"ContainerDied","Data":"8d559bb25acd6e418365c7307fa5788e3524816cf14acb53956d46cc1b3050e6"}
Jul 17 10:23:56.768268 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:56.767909    2556 scope.go:117] "RemoveContainer" containerID="d9fd400efdc8bd87a7c19bab024ee61e40d0e789e04a9b718441e9d4be2713c8"
Jul 17 10:23:56.768268 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:56.768159    2556 scope.go:117] "RemoveContainer" containerID="8d559bb25acd6e418365c7307fa5788e3524816cf14acb53956d46cc1b3050e6"
Jul 17 10:23:56.768413 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:23:56.768374    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8_odh-ai-gateway-infra(be78fa2c-49f1-4eb7-a94f-189c746ca556)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556"
Jul 17 10:23:56.808695 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:56.808667    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57468f77d6-qnx7c_f78bfc57-84aa-48a7-910f-24de74875b54/router/0.log"
Jul 17 10:23:57.792814 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:57.792779    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"]
Jul 17 10:23:57.812634 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:57.812603    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-n2dtr_33ef5c7a-9f1c-4175-8d10-a8f95053d113/main/0.log"
Jul 17 10:23:57.916917 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:57.916886    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:58.072697 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.072592    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be78fa2c-49f1-4eb7-a94f-189c746ca556-maas-api-service-ca\") pod \"be78fa2c-49f1-4eb7-a94f-189c746ca556\" (UID: \"be78fa2c-49f1-4eb7-a94f-189c746ca556\") "
Jul 17 10:23:58.073011 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.072990    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be78fa2c-49f1-4eb7-a94f-189c746ca556-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "be78fa2c-49f1-4eb7-a94f-189c746ca556" (UID: "be78fa2c-49f1-4eb7-a94f-189c746ca556"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:58.173980 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.173944    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be78fa2c-49f1-4eb7-a94f-189c746ca556-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:23:58.779990 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.779907    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"
Jul 17 10:23:58.779990 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.779925    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8" event={"ID":"be78fa2c-49f1-4eb7-a94f-189c746ca556","Type":"ContainerDied","Data":"ae3a6525d06c4206b43e250f99ca30697ff8da399051006f995b2cf968c9c1ac"}
Jul 17 10:23:58.780216 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.779992    2556 scope.go:117] "RemoveContainer" containerID="8d559bb25acd6e418365c7307fa5788e3524816cf14acb53956d46cc1b3050e6"
Jul 17 10:23:58.797948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.797816    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"]
Jul 17 10:23:58.804010 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:23:58.803984    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ltmc8"]
Jul 17 10:24:00.494962 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:00.494910    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" path="/var/lib/kubelet/pods/be78fa2c-49f1-4eb7-a94f-189c746ca556/volumes"
Jul 17 10:24:03.490190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:03.490156    2556 scope.go:117] "RemoveContainer" containerID="68eeadec525cd9ca0b46dac2e4dbe4b9e5c5eacf86abfc2304b5c55e7a245e3c"
Jul 17 10:24:03.810190 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:03.810106    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" event={"ID":"5dae72d0-6a02-42a0-913a-2d71d19f1994","Type":"ContainerStarted","Data":"13d962bd673701484c40d76e12fd22a90d0bd9b65af12f18ade08fc53a6a807c"}
Jul 17 10:24:03.826245 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:03.826190    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" podStartSLOduration=15.826175254 podStartE2EDuration="15.826175254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:23:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:24:03.824548608 +0000 UTC m=+3427.834672533" watchObservedRunningTime="2026-07-17 10:24:03.826175254 +0000 UTC m=+3427.836299177"
Jul 17 10:24:04.269473 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269433    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5lvm5/must-gather-67rl2"]
Jul 17 10:24:04.269776 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269762    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269777    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269788    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269793    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269806    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269816    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269836    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:24:04.269864 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269842    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:24:04.270084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269907    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.270084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269914    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.270084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.269922    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c27ec7-453a-47a6-ae8c-abdfbc4b8dec" containerName="revoke-keys"
Jul 17 10:24:04.270084 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.270020    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="be78fa2c-49f1-4eb7-a94f-189c746ca556" containerName="revoke-keys"
Jul 17 10:24:04.272971 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.272954    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.275535 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.275507    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5lvm5\"/\"kube-root-ca.crt\""
Jul 17 10:24:04.275664 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.275576    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5lvm5\"/\"openshift-service-ca.crt\""
Jul 17 10:24:04.287213 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.287183    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lvm5/must-gather-67rl2"]
Jul 17 10:24:04.291361 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.291336    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:24:04.381313 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.381279    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log" limit=1024
Jul 17 10:24:04.432270 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.432234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7zk\" (UniqueName: \"kubernetes.io/projected/721756c6-8369-4f99-9cd1-f65a073c6fdb-kube-api-access-lx7zk\") pod \"must-gather-67rl2\" (UID: \"721756c6-8369-4f99-9cd1-f65a073c6fdb\") " pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.432458 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.432288    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/721756c6-8369-4f99-9cd1-f65a073c6fdb-must-gather-output\") pod \"must-gather-67rl2\" (UID: \"721756c6-8369-4f99-9cd1-f65a073c6fdb\") " pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.533460 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.533426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7zk\" (UniqueName: \"kubernetes.io/projected/721756c6-8369-4f99-9cd1-f65a073c6fdb-kube-api-access-lx7zk\") pod \"must-gather-67rl2\" (UID: \"721756c6-8369-4f99-9cd1-f65a073c6fdb\") " pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.533460 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.533477    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/721756c6-8369-4f99-9cd1-f65a073c6fdb-must-gather-output\") pod \"must-gather-67rl2\" (UID: \"721756c6-8369-4f99-9cd1-f65a073c6fdb\") " pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.534007 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.533856    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/721756c6-8369-4f99-9cd1-f65a073c6fdb-must-gather-output\") pod \"must-gather-67rl2\" (UID: \"721756c6-8369-4f99-9cd1-f65a073c6fdb\") " pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.542486 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.542458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7zk\" (UniqueName: \"kubernetes.io/projected/721756c6-8369-4f99-9cd1-f65a073c6fdb-kube-api-access-lx7zk\") pod \"must-gather-67rl2\" (UID: \"721756c6-8369-4f99-9cd1-f65a073c6fdb\") " pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.582261 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.582230    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lvm5/must-gather-67rl2"
Jul 17 10:24:04.720870 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.720835    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lvm5/must-gather-67rl2"]
Jul 17 10:24:04.724377 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:24:04.724333    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod721756c6_8369_4f99_9cd1_f65a073c6fdb.slice/crio-331f31b26e7374007aea76f6dd979da7d1800254a6353ada93fb06f84561f23f WatchSource:0}: Error finding container 331f31b26e7374007aea76f6dd979da7d1800254a6353ada93fb06f84561f23f: Status 404 returned error can't find the container with id 331f31b26e7374007aea76f6dd979da7d1800254a6353ada93fb06f84561f23f
Jul 17 10:24:04.817095 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.817054    2556 generic.go:356] "Generic (PLEG): container finished" podID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerID="13d962bd673701484c40d76e12fd22a90d0bd9b65af12f18ade08fc53a6a807c" exitCode=6
Jul 17 10:24:04.817253 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.817117    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" event={"ID":"5dae72d0-6a02-42a0-913a-2d71d19f1994","Type":"ContainerDied","Data":"13d962bd673701484c40d76e12fd22a90d0bd9b65af12f18ade08fc53a6a807c"}
Jul 17 10:24:04.817253 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.817184    2556 scope.go:117] "RemoveContainer" containerID="68eeadec525cd9ca0b46dac2e4dbe4b9e5c5eacf86abfc2304b5c55e7a245e3c"
Jul 17 10:24:04.817465 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.817442    2556 scope.go:117] "RemoveContainer" containerID="13d962bd673701484c40d76e12fd22a90d0bd9b65af12f18ade08fc53a6a807c"
Jul 17 10:24:04.817703 ip-10-0-141-244 kubenswrapper[2556]: E0717 10:24:04.817683    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9_odh-ai-gateway-infra(5dae72d0-6a02-42a0-913a-2d71d19f1994)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994"
Jul 17 10:24:04.818380 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.818352    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/must-gather-67rl2" event={"ID":"721756c6-8369-4f99-9cd1-f65a073c6fdb","Type":"ContainerStarted","Data":"331f31b26e7374007aea76f6dd979da7d1800254a6353ada93fb06f84561f23f"}
Jul 17 10:24:04.833703 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:04.833672    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"]
Jul 17 10:24:05.979761 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:05.979739    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:24:06.148787 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.148756    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dae72d0-6a02-42a0-913a-2d71d19f1994-maas-api-service-ca\") pod \"5dae72d0-6a02-42a0-913a-2d71d19f1994\" (UID: \"5dae72d0-6a02-42a0-913a-2d71d19f1994\") "
Jul 17 10:24:06.149057 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.149034    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dae72d0-6a02-42a0-913a-2d71d19f1994-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5dae72d0-6a02-42a0-913a-2d71d19f1994" (UID: "5dae72d0-6a02-42a0-913a-2d71d19f1994"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:24:06.250209 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.250164    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dae72d0-6a02-42a0-913a-2d71d19f1994-maas-api-service-ca\") on node \"ip-10-0-141-244.ec2.internal\" DevicePath \"\""
Jul 17 10:24:06.830551 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.830461    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"
Jul 17 10:24:06.830551 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.830496    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9" event={"ID":"5dae72d0-6a02-42a0-913a-2d71d19f1994","Type":"ContainerDied","Data":"2dd4451cc8ae687aa1193e6f5a1b6fbca117c1f2d34547e42581063eb16e70a5"}
Jul 17 10:24:06.830806 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.830584    2556 scope.go:117] "RemoveContainer" containerID="13d962bd673701484c40d76e12fd22a90d0bd9b65af12f18ade08fc53a6a807c"
Jul 17 10:24:06.833559 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.833123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/must-gather-67rl2" event={"ID":"721756c6-8369-4f99-9cd1-f65a073c6fdb","Type":"ContainerStarted","Data":"ce414ad9c582d3f5775e88f5e2b6ca903d8439e8f83717ec758c8e544730dbf9"}
Jul 17 10:24:06.833559 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.833165    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/must-gather-67rl2" event={"ID":"721756c6-8369-4f99-9cd1-f65a073c6fdb","Type":"ContainerStarted","Data":"34cc96f5fef02c5cd100b9e875463f4ecf26a0d29cd47c4d50201ebb9402f7c2"}
Jul 17 10:24:06.847280 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.847247    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"]
Jul 17 10:24:06.849805 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.849781    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9x9l9"]
Jul 17 10:24:06.862176 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:06.862113    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5lvm5/must-gather-67rl2" podStartSLOduration=1.7264957380000001 podStartE2EDuration="2.862092293s" totalImagesPullingTime="1.135596555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:24:06.86162853 +0000 UTC m=+3430.871752445" watchObservedRunningTime="2026-07-17 10:24:06.862092293 +0000 UTC m=+3430.872216218"
Jul 17 10:24:07.526740 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:07.526698    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-r6dj8_00555f83-4bb4-4a88-9bb2-4b427ba80497/global-pull-secret-syncer/0.log"
Jul 17 10:24:07.629521 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:07.629444    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5sv88_cd669c69-0076-4509-b99c-40b3a747700f/konnectivity-agent/0.log"
Jul 17 10:24:07.782364 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:07.782282    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-244.ec2.internal_2b3895fcc55e8580e43648b59a9ffbae/haproxy/0.log"
Jul 17 10:24:08.497388 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:08.497351    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" path="/var/lib/kubelet/pods/5dae72d0-6a02-42a0-913a-2d71d19f1994/volumes"
Jul 17 10:24:13.330115 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:13.330070    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b98754ddd-wkw2s_459e981a-6020-4de5-8706-caf932594fa2/authorino/0.log"
Jul 17 10:24:13.431042 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:13.431000    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mvqhv_e23b0300-48e8-4515-acc2-79019874acea/kuadrant-console-plugin/0.log"
Jul 17 10:24:13.585994 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:13.585910    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bcxhk_2be2706b-989b-40d8-babb-237b01f5c399/limitador/0.log"
Jul 17 10:24:15.650217 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:15.650186    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-zbwzx_63a9ea49-2730-4eba-97cf-7c60358c3489/monitoring-plugin/0.log"
Jul 17 10:24:15.863763 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:15.863712    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l2gjk_ebb15d5e-7a58-43af-ab73-9c7843099256/node-exporter/0.log"
Jul 17 10:24:15.884201 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:15.884168    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l2gjk_ebb15d5e-7a58-43af-ab73-9c7843099256/kube-rbac-proxy/0.log"
Jul 17 10:24:15.907923 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:15.907845    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l2gjk_ebb15d5e-7a58-43af-ab73-9c7843099256/init-textfile/0.log"
Jul 17 10:24:17.643171 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:17.643111    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-fqb94_960c6dec-6cfb-4169-bf6e-2ec010ecae40/networking-console-plugin/0.log"
Jul 17 10:24:18.039525 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.039425    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"]
Jul 17 10:24:18.039979 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.039950    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.039979 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.039976    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040012    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040020    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040031    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040039    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040106    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040117    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.040161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.040127    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dae72d0-6a02-42a0-913a-2d71d19f1994" containerName="revoke-keys"
Jul 17 10:24:18.044296 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.044271    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.046730 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.046706    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5lvm5\"/\"default-dockercfg-vqkh2\""
Jul 17 10:24:18.050926 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.050903    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"]
Jul 17 10:24:18.092483 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.092443    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-proc\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.092685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.092490    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-sys\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.092685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.092516    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-podres\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.092685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.092596    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-lib-modules\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.092685 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.092638    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vplgj\" (UniqueName: \"kubernetes.io/projected/9d48a240-e016-4d11-b0cf-26e88cbd8d63-kube-api-access-vplgj\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.194258 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194216    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-proc\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.194524 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194502    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-sys\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.194664 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-podres\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.194793 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194779    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-lib-modules\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.194960 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194942    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vplgj\" (UniqueName: \"kubernetes.io/projected/9d48a240-e016-4d11-b0cf-26e88cbd8d63-kube-api-access-vplgj\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.195140 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194839    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-podres\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.195264 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-sys\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.195369 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-lib-modules\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.195464 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.194436    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9d48a240-e016-4d11-b0cf-26e88cbd8d63-proc\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.206571 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.206542    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vplgj\" (UniqueName: \"kubernetes.io/projected/9d48a240-e016-4d11-b0cf-26e88cbd8d63-kube-api-access-vplgj\") pod \"perf-node-gather-daemonset-47jks\" (UID: \"9d48a240-e016-4d11-b0cf-26e88cbd8d63\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.359662 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.359572    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.548223 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.548140    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"]
Jul 17 10:24:18.557099 ip-10-0-141-244 kubenswrapper[2556]: W0717 10:24:18.556911    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9d48a240_e016_4d11_b0cf_26e88cbd8d63.slice/crio-cb0587ce1b4c04dfdf477dde650699eeb62e0d2c6cb5207cd1a10c5b72948502 WatchSource:0}: Error finding container cb0587ce1b4c04dfdf477dde650699eeb62e0d2c6cb5207cd1a10c5b72948502: Status 404 returned error can't find the container with id cb0587ce1b4c04dfdf477dde650699eeb62e0d2c6cb5207cd1a10c5b72948502
Jul 17 10:24:18.924403 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.924360    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks" event={"ID":"9d48a240-e016-4d11-b0cf-26e88cbd8d63","Type":"ContainerStarted","Data":"fe4d0df55bfc77a858b4ed32b252383cf00fd65b855db991ee74509f30505fd0"}
Jul 17 10:24:18.924948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.924441    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:18.924948 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.924466    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks" event={"ID":"9d48a240-e016-4d11-b0cf-26e88cbd8d63","Type":"ContainerStarted","Data":"cb0587ce1b4c04dfdf477dde650699eeb62e0d2c6cb5207cd1a10c5b72948502"}
Jul 17 10:24:18.940579 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:18.940511    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks" podStartSLOduration=0.940490095 podStartE2EDuration="940.490095ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:24:18.93936391 +0000 UTC m=+3442.949487834" watchObservedRunningTime="2026-07-17 10:24:18.940490095 +0000 UTC m=+3442.950614016"
Jul 17 10:24:19.455977 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:19.455948    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-xk8ql_169e1f61-4f57-45ed-86f7-c6f2a24e96d0/volume-data-source-validator/0.log"
Jul 17 10:24:20.513168 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:20.513137    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fm4nh_ef680662-689d-4cde-8445-9c1322820a32/dns/0.log"
Jul 17 10:24:20.536116 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:20.536087    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fm4nh_ef680662-689d-4cde-8445-9c1322820a32/kube-rbac-proxy/0.log"
Jul 17 10:24:20.635176 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:20.635147    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rb5ff_01e70b2c-a732-4ba8-b436-3caa5b43ccca/dns-node-resolver/0.log"
Jul 17 10:24:21.259014 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:21.258982    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dqwl4_82773457-e021-4aa2-84e9-7d1fe8045d0c/node-ca/0.log"
Jul 17 10:24:22.489892 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:22.489861    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-jhjq6_b5fb7163-04a7-4c5a-b251-e0c4bd52f927/kube-auth-proxy/0.log"
Jul 17 10:24:22.677466 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:22.677430    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57468f77d6-qnx7c_f78bfc57-84aa-48a7-910f-24de74875b54/router/0.log"
Jul 17 10:24:23.373242 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:23.373198    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pt9b_5ccac045-3412-42bb-9aad-ec15a86d870d/serve-healthcheck-canary/0.log"
Jul 17 10:24:24.024481 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:24.024438    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgxft_914869ab-9df5-45b6-962a-e98f8804de31/kube-rbac-proxy/0.log"
Jul 17 10:24:24.047928 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:24.047900    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgxft_914869ab-9df5-45b6-962a-e98f8804de31/exporter/0.log"
Jul 17 10:24:24.071927 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:24.071896    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jgxft_914869ab-9df5-45b6-962a-e98f8804de31/extractor/0.log"
Jul 17 10:24:24.942528 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:24.942494    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-47jks"
Jul 17 10:24:26.606320 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:26.606274    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-2q7sh_7fd30b3c-b529-4a08-a003-a63d1f4a002a/manager/0.log"
Jul 17 10:24:26.632598 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:26.632573    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-2q7sh_7fd30b3c-b529-4a08-a003-a63d1f4a002a/copy-manifests/0.log"
Jul 17 10:24:28.263158 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:28.263108    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7df87f8d9d-mdwn6_597954db-f1e2-4cea-b22e-40826bd0fb86/manager/0.log"
Jul 17 10:24:28.645171 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:28.645127    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-jhjq6_b5fb7163-04a7-4c5a-b251-e0c4bd52f927/kube-auth-proxy/0.log"
Jul 17 10:24:28.774388 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:28.774355    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57468f77d6-qnx7c_f78bfc57-84aa-48a7-910f-24de74875b54/router/0.log"
Jul 17 10:24:30.239702 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:30.239620    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-jhjq6_b5fb7163-04a7-4c5a-b251-e0c4bd52f927/kube-auth-proxy/0.log"
Jul 17 10:24:30.367353 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:30.367321    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57468f77d6-qnx7c_f78bfc57-84aa-48a7-910f-24de74875b54/router/0.log"
Jul 17 10:24:35.120372 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.120344    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/kube-multus-additional-cni-plugins/0.log"
Jul 17 10:24:35.143674 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.143620    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/egress-router-binary-copy/0.log"
Jul 17 10:24:35.168113 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.168084    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/cni-plugins/0.log"
Jul 17 10:24:35.191240 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.191207    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/bond-cni-plugin/0.log"
Jul 17 10:24:35.215763 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.215727    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/routeoverride-cni/0.log"
Jul 17 10:24:35.240496 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.240455    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/whereabouts-cni-bincopy/0.log"
Jul 17 10:24:35.263161 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.263135    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kf66q_35ca4b27-a6bf-445d-8525-9b93ec8ac565/whereabouts-cni/0.log"
Jul 17 10:24:35.378771 ip-10-0-141-244 kubenswrapper[2556]: I0717 10:24:35.378687    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xqrvw_b3ebbbf6-5a57-4076-9df0-378c9aaefb78/kube-multus/0.log"