Jul 09 16:17:39.532123 ip-10-0-139-176 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 16:17:39.532134 ip-10-0-139-176 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 16:17:39.532144 ip-10-0-139-176 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 16:17:39.532615 ip-10-0-139-176 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 16:17:49.636897 ip-10-0-139-176 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 16:17:49.636914 ip-10-0-139-176 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c9db4acc705940e3867cff163447cc99 --
Jul 09 16:19:48.957131 ip-10-0-139-176 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 16:19:50.492786 ip-10-0-139-176 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 16:19:50.492786 ip-10-0-139-176 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 16:19:50.492786 ip-10-0-139-176 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 16:19:50.492786 ip-10-0-139-176 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 16:19:50.492786 ip-10-0-139-176 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 16:19:50.495452 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.495336    2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 16:19:50.499227 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499201    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 16:19:50.499227 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499221    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 16:19:50.499227 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499226    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 16:19:50.499227 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499231    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 16:19:50.499227 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499235    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499239    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499244    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499250    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499256    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499261    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499266    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499270    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499276    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499281    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499286    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499290    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499294    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499299    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499303    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499307    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499319    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499324    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499328    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 16:19:50.499549 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499333    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499337    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499341    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499346    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499350    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499355    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499359    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499363    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499370    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499377    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499381    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499388    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499392    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499396    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499401    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499408    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499413    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499418    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499422    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499426    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 16:19:50.500402 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499431    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499436    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499441    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499445    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499450    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499455    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499459    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499464    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499469    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499474    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499478    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499484    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499488    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499493    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499498    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499503    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499508    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499515    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499522    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 16:19:50.501309 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499527    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499532    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499537    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499542    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499547    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499554    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499558    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499563    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499569    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499574    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499579    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499584    2586 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499589    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499593    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499598    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499602    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499607    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499611    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499615    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499621    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 16:19:50.502161 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499625    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499630    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499635    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499640    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499645    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499649    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499654    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499658    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499663    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499667    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499672    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499677    2586 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499681    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499687    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499691    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499696    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.499700    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501070    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501082    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501088    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501094    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 16:19:50.503145 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501100    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501105    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501111    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501115    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501120    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501125    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501130    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501135    2586 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501140    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501144    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501152    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501158    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501163    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501168    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501173    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501177    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501182    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501187    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501192    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 16:19:50.503964 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501196    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501201    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501206    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501210    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501215    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501219    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501223    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501228    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501235    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501242    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501247    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501252    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501257    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501262    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501266    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501271    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501276    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501280    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501285    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 16:19:50.504499 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501290    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501295    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501302    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501308    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501313    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501317    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501322    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501327    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501332    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501337    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501342    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501348    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501355    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501360    2586 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501365    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501369    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501375    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501380    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501384    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 16:19:50.505004 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501388    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501393    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501401    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501405    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501411    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501415    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501420    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501424    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501429    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501433    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501438    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501443    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501448    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501453    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501457    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501462    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501466    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501471    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501476    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501481    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 16:19:50.505503 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501485    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501490    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501494    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501499    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501506    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501511    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501515    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501520    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501524    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501529    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501533    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501539    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501543    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501548    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501552    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501557    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501562    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.501566    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502318    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502328    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 16:19:50.506066 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502333    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502338    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502344    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502349    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502353    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502358    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502363    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502367    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502372    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502377    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502382    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502386    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502392    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502396    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502401    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502406    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502411    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502416    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502420    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 16:19:50.506605 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502425    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502430    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502434    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502439    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502444    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502448    2586 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502453    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502457    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502462    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502467    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502472    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502477    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502481    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502486    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502490    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502495    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502500    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502508    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502517    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 16:19:50.507122 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502522    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502526    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502532    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502536    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502541    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502546    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502550    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502555    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502559    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502564    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502569    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502574    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502579    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502584    2586 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502589    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502594    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502599    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502604    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502610    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 16:19:50.507627 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502615    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502619    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502623    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502628    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502632    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502637    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502641    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502645    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502650    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502654    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502659    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502664    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502669    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502673    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502678    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502683    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502687    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502692    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502696    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502701    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 16:19:50.508147 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502705    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502710    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502714    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502718    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502723    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502728    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502732    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502738    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502743    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502751    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502756    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502761    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502768    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502775    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502779    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502785    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502790    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502795    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502799    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.502804    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.502948    2586 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 16:19:50.508663 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.502961    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.502971    2586 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.502979    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.502988    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.502994    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503002    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503009    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503015    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503020    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503027    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503033    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503039    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503044    2586 flags.go:64] FLAG: --cgroup-root=""
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503050    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503058    2586 flags.go:64] FLAG: --client-ca-file=""
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503063    2586 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503068    2586 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503082    2586 flags.go:64] FLAG: --cluster-domain=""
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503087    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503093    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503101    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503107    2586 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503114    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 16:19:50.509230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503119    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503124    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503130    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503136    2586 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503141    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503146    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503151    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503156    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503163    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503168    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503173    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503178    2586 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503183    2586 flags.go:64] FLAG: --enable-server="true"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503189    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503197    2586 flags.go:64] FLAG: --event-burst="100"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503203    2586 flags.go:64] FLAG: --event-qps="50"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503207    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503212    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503218    2586 flags.go:64] FLAG: --eviction-hard=""
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503227    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503233    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503238    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503244    2586 flags.go:64] FLAG: --eviction-soft=""
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503251    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503257    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 16:19:50.509826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503261    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503267    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503272    2586 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503277    2586 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503284    2586 flags.go:64] FLAG: --feature-gates=""
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503291    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503296    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503301    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503307    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503313    2586 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503318    2586 flags.go:64] FLAG: --help="false"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503324    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503330    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503335    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503340    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503346    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503352    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503357    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503362    2586 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503367    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503372    2586 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503378    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503383    2586 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503388    2586 flags.go:64] FLAG: --kube-reserved=""
Jul 09 16:19:50.510564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503393    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503398    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503404    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503409    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503413    2586 flags.go:64] FLAG: --lock-file=""
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503418    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503426    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503432    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503442    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503447    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503453    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503458    2586 flags.go:64] FLAG: --logging-format="text"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503465    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503471    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503477    2586 flags.go:64] FLAG: --manifest-url=""
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503482    2586 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503490    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503496    2586 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503503    2586 flags.go:64] FLAG: --max-pods="110"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503508    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503514    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503519    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503525    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503530    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503535    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 16:19:50.511224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503541    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503553    2586 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503559    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503564    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503570    2586 flags.go:64] FLAG: --pod-cidr=""
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503575    2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503585    2586 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503590    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503596    2586 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503601    2586 flags.go:64] FLAG: --port="10250"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503608    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503613    2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-081657ce7d81cfc67"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503618    2586 flags.go:64] FLAG: --qos-reserved=""
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503624    2586 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503631    2586 flags.go:64] FLAG: --register-node="true"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503636    2586 flags.go:64] FLAG: --register-with-taints=""
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503642    2586 flags.go:64] FLAG: --registry-burst="10"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503647    2586 flags.go:64] FLAG: --registry-qps="5"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503652    2586 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503659    2586 flags.go:64] FLAG: --reserved-memory=""
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503666    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503672    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503677    2586 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503682    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503687    2586 flags.go:64] FLAG: --runonce="false"
Jul 09 16:19:50.511888 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503732    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503746    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503751    2586 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503756    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503760    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503776    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503781    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503784    2586 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503787    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503791    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503794    2586 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503797    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503801    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503804    2586 flags.go:64] FLAG: --system-cgroups=""
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503808    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503820    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503824    2586 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503827    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503834    2586 flags.go:64] FLAG: --tls-min-version=""
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503837    2586 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503841    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503845    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503848    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503852    2586 flags.go:64] FLAG: --v="2"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503858    2586 flags.go:64] FLAG: --version="false"
Jul 09 16:19:50.512551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503863    2586 flags.go:64] FLAG: --vmodule=""
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503868    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.503872    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504008    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504013    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504017    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504021    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504025    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504030    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504033    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504036    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504039    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504042    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504045    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504048    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504051    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504054    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504057    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504061    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504063    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 16:19:50.513656 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504066    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504069    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504072    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504074    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504077    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504080    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504083    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504086    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504089    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504091    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504094    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504097    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504100    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504103    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504105    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504108    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504111    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504115    2586 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504121    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504126    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 16:19:50.514367 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504129    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504132    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504136    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504138    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504141    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504144    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504147    2586 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504149    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504152    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504155    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504159    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504163    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504166    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504169    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504172    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504176    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504180    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504184    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504187    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 16:19:50.514918 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504190    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504194    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504197    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504200    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504203    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504206    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504209    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504212    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504216    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504219    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504223    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504227    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504232    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504235    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504238    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504242    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504245    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504248    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504251    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504254    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 16:19:50.515425 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504256    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504259    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504262    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504265    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504267    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504270    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504273    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504276    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504279    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504281    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504284    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504287    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504290    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504293    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504296    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504299    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504302    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504304    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504307    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504310    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 16:19:50.515966 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504313    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504316    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.504319    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.504327    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.511647    2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.511666    2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.511688    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.511698    2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511764    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511771    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511775    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511779    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511783    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511788    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511792    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 16:19:50.516526 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511795    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511798    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511802    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511804    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511807    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511810    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511813    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511816    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511820    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511822    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511826    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511828    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511831    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511854    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511858    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511861    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511864    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511868    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511870    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511874    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 16:19:50.517080 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511876    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511880    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511884    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511887    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511890    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511894    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511897    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511901    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511904    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511907    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511910    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511913    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511916    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511919    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511921    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511924    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511927    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511930    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511933    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511936    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 16:19:50.517602 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511939    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511943    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511947    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511950    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511953    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511956    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511960    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511963    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511967    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511970    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511973    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511976    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511979    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511982    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511985    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511989    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511994    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.511997    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512001    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 16:19:50.518148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512004    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512008    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512011    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512013    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512016    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512019    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512021    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512024    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512027    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512030    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512033    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512035    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512038    2586 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512041    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512044    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512047    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512049    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512052    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512055    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512059    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 16:19:50.518651 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512062    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512065    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512067    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512070    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512073    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512076    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512079    2586 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512082    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512084    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512087    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512090    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512093    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512096    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.512102    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512266    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 16:19:50.519182 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512273    2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512277    2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512281    2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512284    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512287    2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512291    2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512293    2586 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512296    2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512299    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512302    2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512305    2586 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512308    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512311    2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512314    2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512316    2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512319    2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512322    2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512325    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512327    2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512330    2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 16:19:50.519590 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512333    2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512335    2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512338    2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512341    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512344    2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512347    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512350    2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512353    2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512355    2586 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512358    2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512362    2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512365    2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512368    2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512371    2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512373    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512376    2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512379    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512382    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512385    2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512387    2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 16:19:50.520146 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512390    2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512393    2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512396    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512399    2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512401    2586 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512404    2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512407    2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512409    2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512412    2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512415    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512417    2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512420    2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512423    2586 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512425    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512428    2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512430    2586 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512433    2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512436    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512439    2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512441    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 16:19:50.520670 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512444    2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512447    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512450    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512454    2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512456    2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512459    2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512462    2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512464    2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512467    2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512470    2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512473    2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512476    2586 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512479    2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512482    2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512485    2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512487    2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512490    2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512493    2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512495    2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 16:19:50.521198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512498    2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512502    2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512506    2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512509    2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512512    2586 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512516    2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512520    2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512522    2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512525    2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512528    2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512531    2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512534    2586 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512536    2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512539    2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512543    2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512547    2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512550    2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.512600    2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 16:19:50.521685 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:50.514188    2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.514206    2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.514405    2586 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.516794    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.518740    2586 server.go:1019] "Starting client certificate rotation"
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.518876    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.518918    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 16:19:50.522210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.521609    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 16:19:50.523248 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.523234    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 16:19:50.539581 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.539556    2586 log.go:25] "Validated CRI v1 runtime API"
Jul 09 16:19:50.541859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.541827    2586 log.go:25] "Validated CRI v1 image API"
Jul 09 16:19:50.542719 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.542706    2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 16:19:50.545414 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.545393    2586 fs.go:135] Filesystem UUIDs: map[1d48a5ab-37f1-4f19-ab41-e463e5f800c4:/dev/nvme0n1p4 41db3573-2f26-4d32-8b55-f814575eab03:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 09 16:19:50.545465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.545415    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 16:19:50.548464 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.548441    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 16:19:50.552697 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.552561    2586 manager.go:216] Machine: {Timestamp:2026-07-09 16:19:50.550159266 +0000 UTC m=+1.476113538 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3126694 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2909121503166359e9a2700eb6b1f9 SystemUUID:ec290912-1503-1663-59e9-a2700eb6b1f9 BootID:c9db4acc-7059-40e3-867c-ff163447cc99 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3c:ee:ce:b6:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3c:ee:ce:b6:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:45:6e:f1:f8:5a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 16:19:50.552697 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.552690    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 16:19:50.552807 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.552792    2586 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 16:19:50.555939 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.555903    2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 16:19:50.556121 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.555942    2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-176.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 16:19:50.556763 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.556752    2586 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 16:19:50.556796 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.556766    2586 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 16:19:50.556796 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.556774    2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 16:19:50.556796 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.556793    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 16:19:50.558110 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.558098    2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 16:19:50.561222 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561206    2586 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 16:19:50.561403 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561391    2586 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 16:19:50.561454 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561428    2586 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 16:19:50.561485 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561454    2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 16:19:50.561485 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561475    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 16:19:50.561561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561489    2586 kubelet.go:398] "Adding apiserver pod source"
Jul 09 16:19:50.561605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.561564    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 16:19:50.563618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.563585    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 16:19:50.563736 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.563627    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 16:19:50.567594 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.567568    2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 16:19:50.569529 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569513    2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 16:19:50.569583 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569547    2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 16:19:50.569742 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569731    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 16:19:50.569777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569748    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 16:19:50.569777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569755    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 16:19:50.569777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569761    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 16:19:50.569777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569771    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 16:19:50.569909 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.569780    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 16:19:50.570479 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.570470    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 16:19:50.570516 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.570480    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 16:19:50.570516 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.570488    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 16:19:50.570516 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.570494    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 16:19:50.570516 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.570503    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 16:19:50.570516 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.570512    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 16:19:50.571298 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.571288    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 16:19:50.571336 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.571298    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 16:19:50.574260 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.574231    2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-176.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 16:19:50.574895 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.574875    2586 server.go:1268] "Started kubelet"
Jul 09 16:19:50.575013 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.574969    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k256s"
Jul 09 16:19:50.575105 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.575076    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 16:19:50.575105 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.575076    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-176.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 16:19:50.575240 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.575081    2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 16:19:50.575240 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.575184    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 16:19:50.575240 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.575236    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 16:19:50.576364 ip-10-0-139-176 systemd[1]: Started Kubernetes Kubelet.
Jul 09 16:19:50.576958 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.576718    2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 16:19:50.577616 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.577595    2586 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 16:19:50.580929 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.580908    2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 16:19:50.581865 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.581844    2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 16:19:50.581955 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.581912    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 16:19:50.582075 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.582056    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:50.582613 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.582483    2586 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 16:19:50.582613 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.582507    2586 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 16:19:50.582613 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.582489    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 16:19:50.582825 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.582630    2586 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 16:19:50.582825 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.582640    2586 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 16:19:50.582946 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.581709    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-176.ec2.internal.18c0ab7974a16db1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-176.ec2.internal,UID:ip-10-0-139-176.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-176.ec2.internal,},FirstTimestamp:2026-07-09 16:19:50.574845361 +0000 UTC m=+1.500799634,LastTimestamp:2026-07-09 16:19:50.574845361 +0000 UTC m=+1.500799634,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-176.ec2.internal,}"
Jul 09 16:19:50.583059 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583044    2586 factory.go:153] Registering CRI-O factory
Jul 09 16:19:50.583116 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583066    2586 factory.go:223] Registration of the crio container factory successfully
Jul 09 16:19:50.583184 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583167    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 16:19:50.583184 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583178    2586 factory.go:55] Registering systemd factory
Jul 09 16:19:50.583287 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583191    2586 factory.go:223] Registration of the systemd container factory successfully
Jul 09 16:19:50.583287 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583217    2586 factory.go:103] Registering Raw factory
Jul 09 16:19:50.583287 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583234    2586 manager.go:1196] Started watching for new ooms in manager
Jul 09 16:19:50.583582 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.583562    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k256s"
Jul 09 16:19:50.584641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.584617    2586 manager.go:349] Starting recovery of all containers
Jul 09 16:19:50.585821 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.585701    2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-176.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 16:19:50.585909 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.585885    2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 16:19:50.596428 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.596283    2586 manager.go:354] Recovery completed
Jul 09 16:19:50.598167 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.598142    2586 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 09 16:19:50.601672 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.601654    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.603651 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.603628    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.603746 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.603668    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.603746 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.603681    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.604323 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.604305    2586 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 16:19:50.604323 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.604319    2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 16:19:50.604454 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.604333    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.604454 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.604338    2586 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 16:19:50.605111 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.605096    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.605162 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.605124    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.605162 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.605135    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.605422 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.605410    2586 policy_none.go:49] "None policy: Start"
Jul 09 16:19:50.605455 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.605426    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 16:19:50.605455 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.605437    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 16:19:50.606396 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.606384    2586 policy_none.go:47] "Start"
Jul 09 16:19:50.647500 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.647479    2586 manager.go:337] "Starting Device Plugin manager"
Jul 09 16:19:50.647643 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.647526    2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 16:19:50.647643 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.647541    2586 server.go:87] "Starting device plugin registration server"
Jul 09 16:19:50.647876 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.647856    2586 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 16:19:50.647988 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.647875    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 16:19:50.647988 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.647970    2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 16:19:50.648090 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.648062    2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 16:19:50.648090 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.648072    2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 16:19:50.648705 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.648685    2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 16:19:50.648811 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.648740    2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:50.750824 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.750729    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.752099 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.752076    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.752168 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.752115    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.752168 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.752128    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.752168 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.752160    2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.761028 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.761001    2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.761127 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.761036    2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-176.ec2.internal\": node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:50.764508 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.764470    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 16:19:50.765931 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.765901    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 16:19:50.766030 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.765953    2586 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 16:19:50.766030 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.765985    2586 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 16:19:50.766123 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.766028    2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 16:19:50.769745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.769723    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 16:19:50.776776 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.776755    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:50.867165 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.867117    2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"]
Jul 09 16:19:50.867371 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.867210    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.868330 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.868315    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.868389 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.868348    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.868389 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.868360    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.869727 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.869713    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.869886 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.869868    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.869934 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.869905    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.870438 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.870421    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.870525 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.870455    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.870525 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.870466    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.870632 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.870560    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.870632 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.870588    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.870632 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.870605    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.872572 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.872553    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.872679 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.872586    2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 16:19:50.873355 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.873337    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 16:19:50.873442 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.873365    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 16:19:50.873442 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.873376    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeHasSufficientPID"
Jul 09 16:19:50.876825 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.876800    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:50.885015 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.884991    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05ff5927c1fde4b3df7bf6c1bc3393e1-config\") pod \"kube-apiserver-proxy-ip-10-0-139-176.ec2.internal\" (UID: \"05ff5927c1fde4b3df7bf6c1bc3393e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.885111 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.885021    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6569af2e6c8c250ac77f908ed7052815-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal\" (UID: \"6569af2e6c8c250ac77f908ed7052815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.885111 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.885041    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6569af2e6c8c250ac77f908ed7052815-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal\" (UID: \"6569af2e6c8c250ac77f908ed7052815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.902980 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.902951    2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-176.ec2.internal\" not found" node="ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.907778 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.907760    2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-176.ec2.internal\" not found" node="ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.977953 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:50.977907    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:50.985914 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.985890    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05ff5927c1fde4b3df7bf6c1bc3393e1-config\") pod \"kube-apiserver-proxy-ip-10-0-139-176.ec2.internal\" (UID: \"05ff5927c1fde4b3df7bf6c1bc3393e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.985990 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.985922    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6569af2e6c8c250ac77f908ed7052815-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal\" (UID: \"6569af2e6c8c250ac77f908ed7052815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.985990 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.985939    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6569af2e6c8c250ac77f908ed7052815-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal\" (UID: \"6569af2e6c8c250ac77f908ed7052815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.985990 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.985968    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6569af2e6c8c250ac77f908ed7052815-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal\" (UID: \"6569af2e6c8c250ac77f908ed7052815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.986098 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.985996    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05ff5927c1fde4b3df7bf6c1bc3393e1-config\") pod \"kube-apiserver-proxy-ip-10-0-139-176.ec2.internal\" (UID: \"05ff5927c1fde4b3df7bf6c1bc3393e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:50.986098 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:50.986010    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6569af2e6c8c250ac77f908ed7052815-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal\" (UID: \"6569af2e6c8c250ac77f908ed7052815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:51.078248 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.078168    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.185894 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.185847    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.205178 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.205144    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:51.210374 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.210355    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:51.286880 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.286828    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.387223 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.387118    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.445971 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.445942    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 16:19:51.488025 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.487994    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.518613 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.518579    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 16:19:51.519229 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.518732    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 16:19:51.519229 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.518751    2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 16:19:51.586885 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.586847    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 16:14:51 +0000 UTC" deadline="2028-04-23 11:56:30.934490176 +0000 UTC"
Jul 09 16:19:51.586885 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.586877    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15691h36m39.347616842s"
Jul 09 16:19:51.589043 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.589016    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.689654 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.689576    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.760748 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:51.760710    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6569af2e6c8c250ac77f908ed7052815.slice/crio-017bcc98dfa3c07c47a2d50b16a50c1f8e280115a3e23d415c2d833110e1ed7f WatchSource:0}: Error finding container 017bcc98dfa3c07c47a2d50b16a50c1f8e280115a3e23d415c2d833110e1ed7f: Status 404 returned error can't find the container with id 017bcc98dfa3c07c47a2d50b16a50c1f8e280115a3e23d415c2d833110e1ed7f
Jul 09 16:19:51.760952 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:51.760935    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ff5927c1fde4b3df7bf6c1bc3393e1.slice/crio-92e6c25e36b2cabc519193abd31e4ac97fa45f74b70541d3bfe8bc9b2ff652bd WatchSource:0}: Error finding container 92e6c25e36b2cabc519193abd31e4ac97fa45f74b70541d3bfe8bc9b2ff652bd: Status 404 returned error can't find the container with id 92e6c25e36b2cabc519193abd31e4ac97fa45f74b70541d3bfe8bc9b2ff652bd
Jul 09 16:19:51.766462 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.766443    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 16:19:51.769092 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.769048    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal" event={"ID":"05ff5927c1fde4b3df7bf6c1bc3393e1","Type":"ContainerStarted","Data":"92e6c25e36b2cabc519193abd31e4ac97fa45f74b70541d3bfe8bc9b2ff652bd"}
Jul 09 16:19:51.770238 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:51.770214    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal" event={"ID":"6569af2e6c8c250ac77f908ed7052815","Type":"ContainerStarted","Data":"017bcc98dfa3c07c47a2d50b16a50c1f8e280115a3e23d415c2d833110e1ed7f"}
Jul 09 16:19:51.790242 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.790206    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.890470 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.890440    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:51.991630 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:51.991535    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:52.039388 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.039363    2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 16:19:52.091994 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.091954    2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-176.ec2.internal\" not found"
Jul 09 16:19:52.123099 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.123065    2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 16:19:52.182926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.182880    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:52.194453 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.194421    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 16:19:52.195511 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.195496    2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal"
Jul 09 16:19:52.209394 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.209372    2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 16:19:52.565533 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.564925    2586 apiserver.go:52] "Watching apiserver"
Jul 09 16:19:52.574330 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.574302    2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 16:19:52.574909 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.574879    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-fn7v8","kube-system/konnectivity-agent-mqdxc","kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49","openshift-cluster-node-tuning-operator/tuned-tlbht","openshift-multus/multus-additional-cni-plugins-v5xwb","openshift-dns/node-resolver-ms7gh","openshift-image-registry/node-ca-rqpgb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal","openshift-multus/multus-q25q9","openshift-multus/network-metrics-daemon-5p8v6","openshift-network-diagnostics/network-check-target-xtjdr","openshift-network-operator/iptables-alerter-7tvtf","openshift-ovn-kubernetes/ovnkube-node-6qf5t"]
Jul 09 16:19:52.578371 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.577874    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.580290 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.580266    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.581263 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.581229    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.581363 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.581297    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vqtwq\""
Jul 09 16:19:52.581475 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.581439    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.582472 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.582450    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.583063 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.583043    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 16:19:52.583785 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.583764    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 16:19:52.583937 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.583916    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.583995 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.583958    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.584063 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.583764    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9wwf2\""
Jul 09 16:19:52.584960 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.584903    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 16:19:52.585111 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.585066    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cgwdj\""
Jul 09 16:19:52.585238 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.585213    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.585238 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.585071    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.587396 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.587373    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 16:19:52.587500 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.587453    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 16:19:52.587676 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.587656    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.587763 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.587664    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 16:19:52.587864 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.587837    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.588026 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.588008    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.588071 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.588038    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 16:19:52.588188 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.588174    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.588353 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.588328    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rlfsb\""
Jul 09 16:19:52.590240 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.590218    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.591348 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.591326    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.591582 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.591562    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r2nlj\""
Jul 09 16:19:52.591652 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.591592    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.592740 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.592722    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.593285 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.593268    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.593359 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.593319    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.595254 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595195    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 16:19:52.595254 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595237    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.595460 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.595429    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:19:52.595586 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595567    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 16:19:52.595639 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595589    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovn-node-metrics-cert\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.595695 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595636    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.595695 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595675    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjtds\" (UniqueName: \"kubernetes.io/projected/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-kube-api-access-cjtds\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.595801 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595790    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-run-netns\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.595879 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595824    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-systemd\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.595879 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595867    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-node-log\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.595988 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595914    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.595988 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.595946    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjk97\" (UniqueName: \"kubernetes.io/projected/ffba687f-78d4-45a2-8110-1df0cdbc4b40-kube-api-access-mjk97\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.596107 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596089    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g25vj\" (UniqueName: \"kubernetes.io/projected/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-kube-api-access-g25vj\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.596167 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596129    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-kubelet\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596222 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596164    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-slash\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596222 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596214    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596325 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596247    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-cni-netd\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596435 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596322    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 16:19:52.596435 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596346    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-env-overrides\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596435 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596380    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-registration-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.596435 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596405    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-device-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.596641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596440    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-sys-fs\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.596641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596475    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-systemd-units\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596564    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.596641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596595    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-socket-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.596641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596628    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovnkube-script-lib\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.596909 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596663    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-iptables-alerter-script\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.596909 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596696    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-var-lib-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597015 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596911    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-ovn\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597015 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596974    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovnkube-config\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597015 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.596379    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dz6mk\""
Jul 09 16:19:52.597325 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597302    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-log-socket\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597495 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597477    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zchr\" (UniqueName: \"kubernetes.io/projected/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-kube-api-access-8zchr\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597550 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597517    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffba687f-78d4-45a2-8110-1df0cdbc4b40-hosts-file\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.597604 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597551    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffba687f-78d4-45a2-8110-1df0cdbc4b40-tmp-dir\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.597659 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597639    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-host-slash\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.597731 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597712    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-etc-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597789 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597772    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.597870 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.597809    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-cni-bin\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.599645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.599625    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 16:19:52.600552 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.600172    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 16:19:52.600552 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.600335    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 16:19:52.600552 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.600407    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zds8c\""
Jul 09 16:19:52.601203 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.601181    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.602942 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.602768    2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 16:19:52.604081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.603781    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.604567 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.604336    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 16:19:52.604660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.604608    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mp5mz\""
Jul 09 16:19:52.605050 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.604831    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 16:19:52.606183 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.606164    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 16:19:52.606414 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.606390    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ffv6z\""
Jul 09 16:19:52.606505 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.606492    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:52.606637 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.606590    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:19:52.609156 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.609136    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:52.609260 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.609199    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:19:52.618403 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.618379    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zggd8"
Jul 09 16:19:52.626552 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.626529    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zggd8"
Jul 09 16:19:52.684199 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.684172    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 16:19:52.698991 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.698953    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-kubelet\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.698991 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699001    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-slash\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699257 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699073    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-slash\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699257 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699082    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-kubelet\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699257 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699132    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-env-overrides\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699257 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699178    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-device-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699257 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699205    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-sys-fs\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699257 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699244    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c547c999-156c-49dc-8be2-d46c83904a56-tmp\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699272    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-systemd-units\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699298    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699323    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-socket-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699349    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-system-cni-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699376    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-sys\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699399    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-lib-modules\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699437    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-var-lib-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699464    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699491    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.699522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699515    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699539    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-multus-certs\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699563    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc2vr\" (UniqueName: \"kubernetes.io/projected/c547c999-156c-49dc-8be2-d46c83904a56-kube-api-access-mc2vr\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699585    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4987e40b-3a77-46bc-9896-90246ddd8ce4-host\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699609    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699651    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tks7j\" (UniqueName: \"kubernetes.io/projected/8cbcc858-6c65-4abf-b5ca-62934d5952c3-kube-api-access-tks7j\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699662    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-socket-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699687    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffba687f-78d4-45a2-8110-1df0cdbc4b40-tmp-dir\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699716    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-host-slash\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699723    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-device-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699743    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-etc-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699780    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-sys-fs\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699785    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysconfig\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699796    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-env-overrides\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699821    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt66d\" (UniqueName: \"kubernetes.io/projected/48466a63-02d5-4325-a3c1-6e561a9a01b1-kube-api-access-dt66d\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.699900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699883    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-systemd-units\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699912    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699942    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovn-node-metrics-cert\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699945    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-host-slash\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699967    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699979    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.699984    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-etc-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700018    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-cnibin\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700046    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-socket-dir-parent\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700078    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56c97\" (UniqueName: \"kubernetes.io/projected/4145b244-d40e-4bf6-b24c-fc0f31e6deef-kube-api-access-56c97\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700108    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700102    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysctl-d\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700155    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700158    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-run\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700197    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-var-lib-openvswitch\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700218    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-run-netns\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700245    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffba687f-78d4-45a2-8110-1df0cdbc4b40-tmp-dir\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.700465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700260    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-run-netns\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700312    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-systemd\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700341    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-node-log\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700447    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700480    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-daemon-config\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700511    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-var-lib-kubelet\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700541    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700541    2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700570    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-hostroot\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700593    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-conf-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700625    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba0bf9e7-94ef-430a-915a-f80caa23e641-kubelet-config\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700660    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-cni-netd\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700681    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-registration-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700703    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-system-cni-dir\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700725    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-cnibin\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700746    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4987e40b-3a77-46bc-9896-90246ddd8ce4-serviceca\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700768    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d34ad955-4c65-4b5c-ac9c-818e4b3a12fd-agent-certs\") pod \"konnectivity-agent-mqdxc\" (UID: \"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd\") " pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.701225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700788    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-os-release\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700828    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-kubernetes\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700854    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovnkube-script-lib\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700879    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-iptables-alerter-script\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700907    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-ovn\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700929    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovnkube-config\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700953    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-k8s-cni-cncf-io\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700979    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-cni-bin\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.700996    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-registration-dir\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701006    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-etc-kubernetes\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701046    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-node-log\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701069    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-modprobe-d\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701622    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovnkube-script-lib\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701622    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-iptables-alerter-script\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701659    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-ovn\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701690    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701710    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-log-socket\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702109 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701733    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-cni-netd\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701739    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zchr\" (UniqueName: \"kubernetes.io/projected/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-kube-api-access-8zchr\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701769    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-cni-multus\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701771    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-run-systemd\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701796    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-systemd\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701824    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c547c999-156c-49dc-8be2-d46c83904a56-etc-tuned\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701869    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-os-release\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701897    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701924    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701969    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffba687f-78d4-45a2-8110-1df0cdbc4b40-hosts-file\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.701995    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4145b244-d40e-4bf6-b24c-fc0f31e6deef-cni-binary-copy\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702020    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-netns\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702043    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-kubelet\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702068    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysctl-conf\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702097    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-cni-bin\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702125    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjtds\" (UniqueName: \"kubernetes.io/projected/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-kube-api-access-cjtds\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702153    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-cni-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.702970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702194    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-host\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702221    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702250    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovnkube-config\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702311    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-cni-bin\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702318    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffba687f-78d4-45a2-8110-1df0cdbc4b40-hosts-file\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702353    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-log-socket\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702392    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.702417    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhgbp\" (UniqueName: \"kubernetes.io/projected/4987e40b-3a77-46bc-9896-90246ddd8ce4-kube-api-access-lhgbp\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.703007    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba0bf9e7-94ef-430a-915a-f80caa23e641-dbus\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.703034    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d34ad955-4c65-4b5c-ac9c-818e4b3a12fd-konnectivity-ca\") pod \"konnectivity-agent-mqdxc\" (UID: \"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd\") " pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.703056    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjk97\" (UniqueName: \"kubernetes.io/projected/ffba687f-78d4-45a2-8110-1df0cdbc4b40-kube-api-access-mjk97\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.703907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.703078    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g25vj\" (UniqueName: \"kubernetes.io/projected/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-kube-api-access-g25vj\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.706325 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.706302    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-ovn-node-metrics-cert\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.723137 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.723018    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zchr\" (UniqueName: \"kubernetes.io/projected/49361ed9-3456-4c1e-aff8-68a5c23fbcbe-kube-api-access-8zchr\") pod \"ovnkube-node-6qf5t\" (UID: \"49361ed9-3456-4c1e-aff8-68a5c23fbcbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.723596 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.723536    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjtds\" (UniqueName: \"kubernetes.io/projected/0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4-kube-api-access-cjtds\") pod \"aws-ebs-csi-driver-node-w5l49\" (UID: \"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.723949 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.723924    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g25vj\" (UniqueName: \"kubernetes.io/projected/f04af617-bb4a-4f76-8b0d-2b9bb7026c9c-kube-api-access-g25vj\") pod \"iptables-alerter-7tvtf\" (UID: \"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c\") " pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.725229 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.724514    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjk97\" (UniqueName: \"kubernetes.io/projected/ffba687f-78d4-45a2-8110-1df0cdbc4b40-kube-api-access-mjk97\") pod \"node-resolver-ms7gh\" (UID: \"ffba687f-78d4-45a2-8110-1df0cdbc4b40\") " pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.804332 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804295    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-k8s-cni-cncf-io\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804341    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-cni-bin\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804373    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-etc-kubernetes\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804399    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-modprobe-d\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804425    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-cni-multus\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804453    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-systemd\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804477    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c547c999-156c-49dc-8be2-d46c83904a56-etc-tuned\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804501    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-os-release\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.804553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804527    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804557    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804582    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4145b244-d40e-4bf6-b24c-fc0f31e6deef-cni-binary-copy\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804606    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-netns\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804630    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-kubelet\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804656    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysctl-conf\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804681    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-cni-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804703    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-host\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804730    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804755    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhgbp\" (UniqueName: \"kubernetes.io/projected/4987e40b-3a77-46bc-9896-90246ddd8ce4-kube-api-access-lhgbp\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804778    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba0bf9e7-94ef-430a-915a-f80caa23e641-dbus\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804802    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d34ad955-4c65-4b5c-ac9c-818e4b3a12fd-konnectivity-ca\") pod \"konnectivity-agent-mqdxc\" (UID: \"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd\") " pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804851    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c547c999-156c-49dc-8be2-d46c83904a56-tmp\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804881    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-system-cni-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804905    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-sys\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804928    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-lib-modules\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804962    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:52.805060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.804983    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805009    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805036    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-multus-certs\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805068    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc2vr\" (UniqueName: \"kubernetes.io/projected/c547c999-156c-49dc-8be2-d46c83904a56-kube-api-access-mc2vr\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805094    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4987e40b-3a77-46bc-9896-90246ddd8ce4-host\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805144    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tks7j\" (UniqueName: \"kubernetes.io/projected/8cbcc858-6c65-4abf-b5ca-62934d5952c3-kube-api-access-tks7j\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805174    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysconfig\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805203    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt66d\" (UniqueName: \"kubernetes.io/projected/48466a63-02d5-4325-a3c1-6e561a9a01b1-kube-api-access-dt66d\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805232    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-cnibin\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805257    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-socket-dir-parent\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805282    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56c97\" (UniqueName: \"kubernetes.io/projected/4145b244-d40e-4bf6-b24c-fc0f31e6deef-kube-api-access-56c97\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805307    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysctl-d\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805331    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-run\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805372    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-daemon-config\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805397    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-var-lib-kubelet\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805425    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-hostroot\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.805926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805463    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-conf-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805489    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba0bf9e7-94ef-430a-915a-f80caa23e641-kubelet-config\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805521    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-system-cni-dir\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805553    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-cnibin\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805582    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4987e40b-3a77-46bc-9896-90246ddd8ce4-serviceca\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805607    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d34ad955-4c65-4b5c-ac9c-818e4b3a12fd-agent-certs\") pod \"konnectivity-agent-mqdxc\" (UID: \"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd\") " pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805631    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-os-release\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805656    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-kubernetes\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805780    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-kubernetes\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805850    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-k8s-cni-cncf-io\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805889    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-cni-bin\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.805924    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-etc-kubernetes\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.806043    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-modprobe-d\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.806084    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-cni-multus\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.806137    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-systemd\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.806859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.806653    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-multus-certs\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807009    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-sys\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807111    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-lib-modules\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807135    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-run-netns\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807234    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-os-release\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807256    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba0bf9e7-94ef-430a-915a-f80caa23e641-dbus\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807502    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.807587 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807565    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-host-var-lib-kubelet\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.807976 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807737    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysctl-conf\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.807976 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.807759    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:52.807976 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807813    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-cni-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.807976 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.807861    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:19:53.307837242 +0000 UTC m=+4.233791524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:52.807976 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.807883    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-host\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.808229 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808061    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-run\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.808229 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.808164    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:52.808229 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.808220    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:19:53.308200432 +0000 UTC m=+4.234154702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:52.808437 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808415    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4145b244-d40e-4bf6-b24c-fc0f31e6deef-cni-binary-copy\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808508 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808462    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d34ad955-4c65-4b5c-ac9c-818e4b3a12fd-konnectivity-ca\") pod \"konnectivity-agent-mqdxc\" (UID: \"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd\") " pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.808570 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-system-cni-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808570 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808550    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-system-cni-dir\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.808679 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808566    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysconfig\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.808679 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808585    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-var-lib-kubelet\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.808679 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808597    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-daemon-config\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808679 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808617    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-hostroot\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808644    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-cnibin\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808668    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48466a63-02d5-4325-a3c1-6e561a9a01b1-cnibin\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808756    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808873    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-socket-dir-parent\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808893    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-multus-conf-dir\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808928    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba0bf9e7-94ef-430a-915a-f80caa23e641-kubelet-config\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:52.808978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.808961    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4987e40b-3a77-46bc-9896-90246ddd8ce4-host\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.809334 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.809002    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4145b244-d40e-4bf6-b24c-fc0f31e6deef-os-release\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.809334 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.809077    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4987e40b-3a77-46bc-9896-90246ddd8ce4-serviceca\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.809334 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.809098    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c547c999-156c-49dc-8be2-d46c83904a56-etc-sysctl-d\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.809542 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.809516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.809622 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.809574    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48466a63-02d5-4325-a3c1-6e561a9a01b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.811945 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.811904    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c547c999-156c-49dc-8be2-d46c83904a56-tmp\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.812070 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.811960    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c547c999-156c-49dc-8be2-d46c83904a56-etc-tuned\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.812143 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.812065    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d34ad955-4c65-4b5c-ac9c-818e4b3a12fd-agent-certs\") pod \"konnectivity-agent-mqdxc\" (UID: \"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd\") " pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.819565 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.819504    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhgbp\" (UniqueName: \"kubernetes.io/projected/4987e40b-3a77-46bc-9896-90246ddd8ce4-kube-api-access-lhgbp\") pod \"node-ca-rqpgb\" (UID: \"4987e40b-3a77-46bc-9896-90246ddd8ce4\") " pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.821755 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.821730    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:19:52.821890 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.821764    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:19:52.821890 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.821780    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:52.821890 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:52.821886    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:19:53.321855083 +0000 UTC m=+4.247809341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:52.824338 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.824295    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56c97\" (UniqueName: \"kubernetes.io/projected/4145b244-d40e-4bf6-b24c-fc0f31e6deef-kube-api-access-56c97\") pod \"multus-q25q9\" (UID: \"4145b244-d40e-4bf6-b24c-fc0f31e6deef\") " pod="openshift-multus/multus-q25q9"
Jul 09 16:19:52.825083 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.824737    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tks7j\" (UniqueName: \"kubernetes.io/projected/8cbcc858-6c65-4abf-b5ca-62934d5952c3-kube-api-access-tks7j\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:52.825083 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.825039    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt66d\" (UniqueName: \"kubernetes.io/projected/48466a63-02d5-4325-a3c1-6e561a9a01b1-kube-api-access-dt66d\") pod \"multus-additional-cni-plugins-v5xwb\" (UID: \"48466a63-02d5-4325-a3c1-6e561a9a01b1\") " pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.825760 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.825737    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc2vr\" (UniqueName: \"kubernetes.io/projected/c547c999-156c-49dc-8be2-d46c83904a56-kube-api-access-mc2vr\") pod \"tuned-tlbht\" (UID: \"c547c999-156c-49dc-8be2-d46c83904a56\") " pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.899564 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.899523    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ms7gh"
Jul 09 16:19:52.904062 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.904038    2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 16:19:52.908520 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.908486    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7tvtf"
Jul 09 16:19:52.913269 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:52.913240    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffba687f_78d4_45a2_8110_1df0cdbc4b40.slice/crio-96db5d0cea77b7641e193537fb0f374db7cff80581bc6f976754ae7997ae7b07 WatchSource:0}: Error finding container 96db5d0cea77b7641e193537fb0f374db7cff80581bc6f976754ae7997ae7b07: Status 404 returned error can't find the container with id 96db5d0cea77b7641e193537fb0f374db7cff80581bc6f976754ae7997ae7b07
Jul 09 16:19:52.920399 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:52.920367    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04af617_bb4a_4f76_8b0d_2b9bb7026c9c.slice/crio-d924382f0ea7f8d9489327c9e91209a5f206130817eefe9fe4d7404d676929b4 WatchSource:0}: Error finding container d924382f0ea7f8d9489327c9e91209a5f206130817eefe9fe4d7404d676929b4: Status 404 returned error can't find the container with id d924382f0ea7f8d9489327c9e91209a5f206130817eefe9fe4d7404d676929b4
Jul 09 16:19:52.926084 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.926066    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:19:52.931194 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.931166    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49"
Jul 09 16:19:52.938421 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.938385    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tlbht"
Jul 09 16:19:52.950089 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.948995    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v5xwb"
Jul 09 16:19:52.957154 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.957131    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rqpgb"
Jul 09 16:19:52.967076 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.967045    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:19:52.974213 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:52.974188    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q25q9"
Jul 09 16:19:53.260956 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.260866    2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 16:19:53.310820 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.310760    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:53.311026 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.310873    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:53.311095 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.311050    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:53.311150 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.311137    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:19:54.311113537 +0000 UTC m=+5.237067820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:53.313913 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.312071    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:53.313913 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.312337    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:19:54.312314816 +0000 UTC m=+5.238269075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:53.411614 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.411563    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:53.411824 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.411751    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:19:53.411824 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.411801    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:19:53.411824 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.411819    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:53.411951 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.411880    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:19:54.411864065 +0000 UTC m=+5.337818322 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:53.632090 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.631924    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 16:14:53 +0000 UTC" deadline="2028-03-27 20:13:00.781797008 +0000 UTC"
Jul 09 16:19:53.632090 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.631987    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15051h53m7.149814802s"
Jul 09 16:19:53.766555 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.766512    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:53.766721 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:53.766637    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:19:53.774246 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.774206    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7tvtf" event={"ID":"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c","Type":"ContainerStarted","Data":"d924382f0ea7f8d9489327c9e91209a5f206130817eefe9fe4d7404d676929b4"}
Jul 09 16:19:53.775511 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:53.775481    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ms7gh" event={"ID":"ffba687f-78d4-45a2-8110-1df0cdbc4b40","Type":"ContainerStarted","Data":"96db5d0cea77b7641e193537fb0f374db7cff80581bc6f976754ae7997ae7b07"}
Jul 09 16:19:54.026851 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:19:54.026710    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49361ed9_3456_4c1e_aff8_68a5c23fbcbe.slice/crio-75d27cce956baab87d53a67fde34cbe8ba6068d8485cb4963e197cf8f18338ed WatchSource:0}: Error finding container 75d27cce956baab87d53a67fde34cbe8ba6068d8485cb4963e197cf8f18338ed: Status 404 returned error can't find the container with id 75d27cce956baab87d53a67fde34cbe8ba6068d8485cb4963e197cf8f18338ed
Jul 09 16:19:54.321183 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.320905    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:54.321435 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.321368    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:54.321496 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.321438    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:19:56.321416447 +0000 UTC m=+7.247370708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:54.321659 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.321628    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:54.321825 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.321803    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:54.321907 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.321881    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:19:56.32186215 +0000 UTC m=+7.247816429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:54.422730 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.422643    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:54.422949 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.422873    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:19:54.422949 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.422907    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:19:54.422949 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.422923    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:54.423122 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.422989    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:19:56.422969453 +0000 UTC m=+7.348923729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:54.632733 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.632110    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 16:14:53 +0000 UTC" deadline="2027-12-31 09:56:54.529103738 +0000 UTC"
Jul 09 16:19:54.632733 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.632154    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12953h36m59.89695512s"
Jul 09 16:19:54.767013 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.766976    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:54.767175 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.767135    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:19:54.767531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.767513    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:54.767626 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:54.767590    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:19:54.782585 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.782549    2586 generic.go:356] "Generic (PLEG): container finished" podID="6569af2e6c8c250ac77f908ed7052815" containerID="ec0e732d2dcf9b541686133bb8886aa7287d4adcd1b777594bc6084c46f67daf" exitCode=0
Jul 09 16:19:54.782764 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.782654    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal" event={"ID":"6569af2e6c8c250ac77f908ed7052815","Type":"ContainerDied","Data":"ec0e732d2dcf9b541686133bb8886aa7287d4adcd1b777594bc6084c46f67daf"}
Jul 09 16:19:54.785224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.785196    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerStarted","Data":"8a8b231c552e704b908dd282d6f5bb379225fb45babaa7c8fbacb965d512a678"}
Jul 09 16:19:54.788867 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.788323    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tlbht" event={"ID":"c547c999-156c-49dc-8be2-d46c83904a56","Type":"ContainerStarted","Data":"efd3654bd0d20f73be793b2d6be9f4bc77e033c09e51846661a6b486b72e25fd"}
Jul 09 16:19:54.801957 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.801921    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal" event={"ID":"05ff5927c1fde4b3df7bf6c1bc3393e1","Type":"ContainerStarted","Data":"9dad697aa7ba7ce23c03144a0f886bd4a14bb21915faa76490db5958314f9f31"}
Jul 09 16:19:54.804031 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.803987    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q25q9" event={"ID":"4145b244-d40e-4bf6-b24c-fc0f31e6deef","Type":"ContainerStarted","Data":"8475386c2922da37de0152a2741dc447d2121f69fd288c98d9a16c841233c37f"}
Jul 09 16:19:54.814688 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.814652    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49" event={"ID":"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4","Type":"ContainerStarted","Data":"f25ed92f6d6641a0e497396741e8b23895511c295327cfc21eac2363f53405ea"}
Jul 09 16:19:54.827018 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.826976    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mqdxc" event={"ID":"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd","Type":"ContainerStarted","Data":"37588b50a0965603dbafed3e66d404b06333da6dfed305b003dbc60504d3711c"}
Jul 09 16:19:54.843532 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.843488    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rqpgb" event={"ID":"4987e40b-3a77-46bc-9896-90246ddd8ce4","Type":"ContainerStarted","Data":"fd67d82a56d5f20e9657468f970076988c75dbc3f3516aad9f2b9de63f26dd6f"}
Jul 09 16:19:54.849104 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:54.849058    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"75d27cce956baab87d53a67fde34cbe8ba6068d8485cb4963e197cf8f18338ed"}
Jul 09 16:19:55.766670 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:55.766623    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:55.767221 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:55.766781    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:19:55.856398 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:55.856315    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal" event={"ID":"6569af2e6c8c250ac77f908ed7052815","Type":"ContainerStarted","Data":"3bd4129c5bc60ff9ed8dd18f4b7f6fd0bd785e0427e70181b274d9ff528eb977"}
Jul 09 16:19:55.872979 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:55.872914    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-176.ec2.internal" podStartSLOduration=3.8728898689999998 podStartE2EDuration="3.872889869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:19:54.818867914 +0000 UTC m=+5.744822196" watchObservedRunningTime="2026-07-09 16:19:55.872889869 +0000 UTC m=+6.798844153"
Jul 09 16:19:55.873493 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:55.873452    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-176.ec2.internal" podStartSLOduration=3.873439957 podStartE2EDuration="3.873439957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:19:55.872469555 +0000 UTC m=+6.798423838" watchObservedRunningTime="2026-07-09 16:19:55.873439957 +0000 UTC m=+6.799394240"
Jul 09 16:19:56.340997 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:56.340946    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:56.341208 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:56.341064    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:56.341276 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.341251    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:56.341345 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.341336    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:00.341302184 +0000 UTC m=+11.267256444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:19:56.341921 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.341897    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:56.342061 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.341980    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:00.34196109 +0000 UTC m=+11.267915351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:19:56.441986 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:56.441943    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:56.442185 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.442137    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:19:56.442185 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.442158    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:19:56.442185 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.442171    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:56.442352 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.442235    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:20:00.442215864 +0000 UTC m=+11.368170147 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:19:56.768641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:56.767984    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:56.768641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:56.768043    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:56.768641 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.768182    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:19:56.768641 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:56.768308    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:19:57.766733 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:57.766695    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:57.766953 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:57.766865    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:19:58.767490 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:58.766925    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:19:58.767490 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:58.767061    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:19:58.768422 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:58.768181    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:19:58.768422 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:58.768372    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:19:59.766473 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:19:59.766435    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:19:59.766677 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:19:59.766558    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:00.377055 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:00.377009    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:00.377527 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:00.377109    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:00.377527 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.377213    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:20:00.377527 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.377273    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:08.37725728 +0000 UTC m=+19.303211536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:20:00.377527 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.377213    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:20:00.377527 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.377342    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:08.37732914 +0000 UTC m=+19.303283412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:20:00.477547 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:00.477452    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:00.477717 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.477662    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:20:00.477717 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.477691    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:20:00.477717 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.477706    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:20:00.477903 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.477772    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:20:08.477751806 +0000 UTC m=+19.403706083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:20:00.767792 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:00.767296    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:00.767792 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.767431    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:00.767792 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:00.767296    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:00.768096 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:00.767931    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:01.766580 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:01.766542    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:01.767163 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:01.766689    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:02.767528 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:02.766805    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:02.767528 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:02.766959    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:02.767528 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:02.767384    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:02.767528 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:02.767480    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:03.766742 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:03.766699    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:03.766905 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:03.766847    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:04.766637 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:04.766605    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:04.767140 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:04.766613    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:04.767140 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:04.766748    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:04.767140 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:04.766927    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:05.767136 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:05.767093    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:05.767558 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:05.767212    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:06.766628 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:06.766586    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:06.766628 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:06.766621    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:06.766859 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:06.766726    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:06.766859 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:06.766839    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:07.766645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:07.766606    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:07.767172 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:07.766741    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:08.441926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:08.441884    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:08.442183 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:08.441970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:08.442183 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.442007    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:20:08.442183 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.442083    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:20:08.442183 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.442086    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:24.442069484 +0000 UTC m=+35.368023760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:20:08.442183 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.442147    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:24.442130734 +0000 UTC m=+35.368084997 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:20:08.542513 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:08.542470    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:08.542697 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.542617    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:20:08.542697 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.542636    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:20:08.542697 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.542649    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:20:08.542882 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.542701    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:20:24.542688237 +0000 UTC m=+35.468642493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:20:08.766899 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:08.766825    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:08.767358 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:08.766825    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:08.767358 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.766931    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:08.767358 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:08.767105    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:09.766367 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:09.766324    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:09.766555 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:09.766456    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:10.767007 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:10.766965    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:10.767007 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:10.766995    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:10.767492 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:10.767096    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:10.767492 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:10.767216    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:11.766298 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:11.766257    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:11.766469 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:11.766382    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:12.767174 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:12.767139    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:12.767711 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:12.767291    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:12.767711 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:12.767338    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:12.767711 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:12.767445    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:13.766733 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.766425    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:13.766986 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:13.766844    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:13.894278 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.894240    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q25q9" event={"ID":"4145b244-d40e-4bf6-b24c-fc0f31e6deef","Type":"ContainerStarted","Data":"bc009f0bd966aabe9b106465948297edb9148f19459841af48aadd7c1187e5be"}
Jul 09 16:20:13.895730 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.895699    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49" event={"ID":"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4","Type":"ContainerStarted","Data":"52114a8f3430f83561044a404c94fc6c4c893c15715ac87e59f38906f7268733"}
Jul 09 16:20:13.897102 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.897081    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mqdxc" event={"ID":"d34ad955-4c65-4b5c-ac9c-818e4b3a12fd","Type":"ContainerStarted","Data":"0b7f4486bd6a785ad1eca219b793a21656e5d55dea637edf028bd584149d05e7"}
Jul 09 16:20:13.898557 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.898530    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rqpgb" event={"ID":"4987e40b-3a77-46bc-9896-90246ddd8ce4","Type":"ContainerStarted","Data":"9c9b066e93ecdb5cb1d8ebf8fec3646e074213e2021ca1daca3b1df0f116ade5"}
Jul 09 16:20:13.901107 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901089    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:20:13.901380 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901365    2586 generic.go:356] "Generic (PLEG): container finished" podID="49361ed9-3456-4c1e-aff8-68a5c23fbcbe" containerID="3c7e9b696a945e5d3d3a7ae7d3282a1fcfafe6a870df48150007dcb3c1c0d560" exitCode=1
Jul 09 16:20:13.901439 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901426    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"48f35c60b7634439b3e0a52efd0f210b5cd6519801c62635a268dc43fc968e5b"}
Jul 09 16:20:13.901491 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901474    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"635444f3f47b6f65583e5f4395d762717e1b36353037535d1c009fd4df58aaf6"}
Jul 09 16:20:13.901539 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901505    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"fae463a4e1b1f31dd1ae005487d7e7a1c8cd9b5d9730814f4732852b0fe5be82"}
Jul 09 16:20:13.901576 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901538    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"ce9114d5fefeeea97b7e251a2c5a78b70759c530459a93f1f964ca52dd5899a2"}
Jul 09 16:20:13.901576 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901566    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerDied","Data":"3c7e9b696a945e5d3d3a7ae7d3282a1fcfafe6a870df48150007dcb3c1c0d560"}
Jul 09 16:20:13.901648 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.901602    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"9c252c2e3a0e3b18436530387c15105855237072eba344e4d9e9b2c29d32f178"}
Jul 09 16:20:13.902654 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.902626    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ms7gh" event={"ID":"ffba687f-78d4-45a2-8110-1df0cdbc4b40","Type":"ContainerStarted","Data":"3d51d70c406f1567222edd05b86f5682733bf61b686cf670ccdbc10b7847e677"}
Jul 09 16:20:13.904063 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.904040    2586 generic.go:356] "Generic (PLEG): container finished" podID="48466a63-02d5-4325-a3c1-6e561a9a01b1" containerID="f991b6203ae8d1830d4b7824ebfc3bfa39180c32acffb80fbb5e7293d2aa9f51" exitCode=0
Jul 09 16:20:13.904158 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.904070    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerDied","Data":"f991b6203ae8d1830d4b7824ebfc3bfa39180c32acffb80fbb5e7293d2aa9f51"}
Jul 09 16:20:13.905401 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.905383    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tlbht" event={"ID":"c547c999-156c-49dc-8be2-d46c83904a56","Type":"ContainerStarted","Data":"2713baa7bc758025199544414f5e45576ed3c62198c0857544b8ecedfc95f826"}
Jul 09 16:20:13.922620 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.922574    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q25q9" podStartSLOduration=3.964353163 podStartE2EDuration="22.922560419s" totalImagesPullingTime="18.958207256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:13.922248202 +0000 UTC m=+24.848202463" watchObservedRunningTime="2026-07-09 16:20:13.922560419 +0000 UTC m=+24.848514697"
Jul 09 16:20:13.951334 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.951280    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mqdxc" podStartSLOduration=4.050975744 podStartE2EDuration="22.951265547s" totalImagesPullingTime="18.900289803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:13.951163435 +0000 UTC m=+24.877117714" watchObservedRunningTime="2026-07-09 16:20:13.951265547 +0000 UTC m=+24.877219824"
Jul 09 16:20:13.978435 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:13.978388    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ms7gh" podStartSLOduration=10.993236667 podStartE2EDuration="22.978372474s" totalImagesPullingTime="11.985135807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:13.97815746 +0000 UTC m=+24.904111741" watchObservedRunningTime="2026-07-09 16:20:13.978372474 +0000 UTC m=+24.904326752"
Jul 09 16:20:14.021542 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:14.021486    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tlbht" podStartSLOduration=4.122875952 podStartE2EDuration="23.0214676s" totalImagesPullingTime="18.898591648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:13.99635129 +0000 UTC m=+24.922305569" watchObservedRunningTime="2026-07-09 16:20:14.0214676 +0000 UTC m=+24.947421941"
Jul 09 16:20:14.041516 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:14.041469    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rqpgb" podStartSLOduration=4.138455993 podStartE2EDuration="23.041457192s" totalImagesPullingTime="18.903001199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:14.041357961 +0000 UTC m=+24.967312239" watchObservedRunningTime="2026-07-09 16:20:14.041457192 +0000 UTC m=+24.967411469"
Jul 09 16:20:14.766957 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:14.766929    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:14.767191 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:14.766963    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:14.767191 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:14.767031    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:14.767191 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:14.767139    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:14.908745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:14.908709    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7tvtf" event={"ID":"f04af617-bb4a-4f76-8b0d-2b9bb7026c9c","Type":"ContainerStarted","Data":"8b57c020605267972f5e2d5edf3cc40c2d9688f9e02633f1fc445bfac4c9b944"}
Jul 09 16:20:14.926077 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:14.926007    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7tvtf" podStartSLOduration=3.9118300660000003 podStartE2EDuration="23.925991034s" totalImagesPullingTime="20.014160968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:14.925667665 +0000 UTC m=+25.851621943" watchObservedRunningTime="2026-07-09 16:20:14.925991034 +0000 UTC m=+25.851945312"
Jul 09 16:20:15.767064 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:15.767023    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:15.767252 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:15.767157    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:15.913049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:15.913011    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49" event={"ID":"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4","Type":"ContainerStarted","Data":"b8d4ada1314ec59ec558f4b870b8a7080af74646fc64eb31f2ac2cd3d0686df8"}
Jul 09 16:20:15.917069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:15.917041    2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 16:20:15.917461 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:15.917441    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:20:15.917902 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:15.917876    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"e52ec8d6937b9dfaab234d6476237d3ee341be9e1fe013180516ccc2e710891b"}
Jul 09 16:20:16.581112 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.580923    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:20:16.581534 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.581505    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:20:16.664895 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.664803    2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T16:20:15.917063715Z","UUID":"2547029b-9c48-4486-bd26-75735c71e7f9","Handler":null,"Name":"","Endpoint":""}
Jul 09 16:20:16.666455 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.666437    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 16:20:16.666573 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.666475    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 16:20:16.766778 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.766732    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:16.766964 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.766795    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:16.766964 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:16.766910    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:16.767084 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:16.767004    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:16.920013 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.919978    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:20:16.920465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:16.920448    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mqdxc"
Jul 09 16:20:17.766573 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:17.766537    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:17.766741 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:17.766643    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:17.924342 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:17.924305    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49" event={"ID":"0453e02f-f7d1-42fe-a42e-ddf8cd0bf8c4","Type":"ContainerStarted","Data":"4f45960e70a36c09377af0975aacba627739adbf8ab2632cd7173b158955f468"}
Jul 09 16:20:17.956402 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:17.956343    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w5l49" podStartSLOduration=4.11766708 podStartE2EDuration="26.956323191s" totalImagesPullingTime="22.838656111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:17.95589005 +0000 UTC m=+28.881844328" watchObservedRunningTime="2026-07-09 16:20:17.956323191 +0000 UTC m=+28.882277471"
Jul 09 16:20:18.767129 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:18.766940    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:18.767322 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:18.767024    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:18.767322 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:18.767205    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:18.767395 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:18.767324    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:18.929806 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:18.929778    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:20:18.930455 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:18.930183    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"8ab2683791a65784c4e2d1d679878dffdbc51189ec6c71d7118f793ced814d24"}
Jul 09 16:20:18.930678 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:18.930660    2586 scope.go:117] "RemoveContainer" containerID="3c7e9b696a945e5d3d3a7ae7d3282a1fcfafe6a870df48150007dcb3c1c0d560"
Jul 09 16:20:19.767269 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.767231    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:19.767426 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:19.767362    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:19.938087 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.938058    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:20:19.938712 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.938673    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" event={"ID":"49361ed9-3456-4c1e-aff8-68a5c23fbcbe","Type":"ContainerStarted","Data":"a45a71c6aed39b3418383f4ebfbe80788898c66660d73ea036e958702f9ac859"}
Jul 09 16:20:19.939245 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.939222    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:20:19.939352 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.939295    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:20:19.939352 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.939330    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:20:19.958893 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.958858    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:20:19.959041 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.959012    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:20:19.971734 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:19.971686    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t" podStartSLOduration=9.998401982 podStartE2EDuration="28.971668605s" totalImagesPullingTime="18.973266623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:19.97073593 +0000 UTC m=+30.896690221" watchObservedRunningTime="2026-07-09 16:20:19.971668605 +0000 UTC m=+30.897622866"
Jul 09 16:20:20.048259 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:20.048138    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xtjdr"]
Jul 09 16:20:20.048413 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:20.048267    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:20.048413 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:20.048376    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:20.052180 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:20.052125    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fn7v8"]
Jul 09 16:20:20.052371 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:20.052253    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:20.052430 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:20.052380    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:20.052881 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:20.052846    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5p8v6"]
Jul 09 16:20:20.053120 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:20.052944    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:20.053120 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:20.053037    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:21.766826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:21.766787    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:21.766826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:21.766820    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:21.767338 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:21.766802    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:21.767338 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:21.766928    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:21.767338 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:21.767021    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:21.767338 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:21.767135    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:22.947503 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:22.947235    2586 generic.go:356] "Generic (PLEG): container finished" podID="48466a63-02d5-4325-a3c1-6e561a9a01b1" containerID="eb16421e90a668ed18f62502cab6f3358da2bb385d6a514d257f6d75c3a02cf3" exitCode=0
Jul 09 16:20:22.947503 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:22.947315    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerDied","Data":"eb16421e90a668ed18f62502cab6f3358da2bb385d6a514d257f6d75c3a02cf3"}
Jul 09 16:20:23.766647 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:23.766611    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:23.766647 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:23.766648    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:23.766897 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:23.766648    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:23.766897 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:23.766728    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:23.766897 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:23.766875    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:23.766999 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:23.766955    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:24.460963 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:24.460861    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:24.460963 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:24.460935    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:24.461580 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.461017    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 16:20:24.461580 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.461050    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:20:24.461580 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.461082    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret podName:ba0bf9e7-94ef-430a-915a-f80caa23e641 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:56.461068142 +0000 UTC m=+67.387022402 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret") pod "global-pull-secret-syncer-fn7v8" (UID: "ba0bf9e7-94ef-430a-915a-f80caa23e641") : object "kube-system"/"original-pull-secret" not registered
Jul 09 16:20:24.461580 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.461099    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs podName:8cbcc858-6c65-4abf-b5ca-62934d5952c3 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:56.461090394 +0000 UTC m=+67.387044649 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs") pod "network-metrics-daemon-5p8v6" (UID: "8cbcc858-6c65-4abf-b5ca-62934d5952c3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 16:20:24.561773 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:24.561738    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:24.561973 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.561912    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 16:20:24.561973 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.561932    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 16:20:24.561973 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.561944    2586 projected.go:196] Error preparing data for projected volume kube-api-access-kl7wn for pod openshift-network-diagnostics/network-check-target-xtjdr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:20:24.562083 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:24.561994    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn podName:97eb91c1-f07b-4165-81b5-586f3f38abeb nodeName:}" failed. No retries permitted until 2026-07-09 16:20:56.561979544 +0000 UTC m=+67.487933800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kl7wn" (UniqueName: "kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn") pod "network-check-target-xtjdr" (UID: "97eb91c1-f07b-4165-81b5-586f3f38abeb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 16:20:24.954241 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:24.954193    2586 generic.go:356] "Generic (PLEG): container finished" podID="48466a63-02d5-4325-a3c1-6e561a9a01b1" containerID="699e2b57169e71c18effef9e3b88308655638097ce3be3143524f5fe4f2d3b56" exitCode=0
Jul 09 16:20:24.954241 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:24.954246    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerDied","Data":"699e2b57169e71c18effef9e3b88308655638097ce3be3143524f5fe4f2d3b56"}
Jul 09 16:20:25.766367 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.766287    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:25.766367 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.766309    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:25.766367 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.766309    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:25.766995 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:25.766406    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5p8v6" podUID="8cbcc858-6c65-4abf-b5ca-62934d5952c3"
Jul 09 16:20:25.766995 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:25.766464    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fn7v8" podUID="ba0bf9e7-94ef-430a-915a-f80caa23e641"
Jul 09 16:20:25.766995 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:25.766535    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xtjdr" podUID="97eb91c1-f07b-4165-81b5-586f3f38abeb"
Jul 09 16:20:25.845576 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.845549    2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-176.ec2.internal" event="NodeReady"
Jul 09 16:20:25.845714 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.845673    2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 16:20:25.897081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.897046    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nzgfq"]
Jul 09 16:20:25.915278 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.915248    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mgpzc"]
Jul 09 16:20:25.915469 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.915383    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:25.918841 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.918814    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 16:20:25.918841 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.918836    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 16:20:25.919039 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.918899    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-466pn\""
Jul 09 16:20:25.925197 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.925174    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzgfq"]
Jul 09 16:20:25.925318 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.925219    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mgpzc"]
Jul 09 16:20:25.925376 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.925314    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:25.926106 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.926082    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9fbtj"]
Jul 09 16:20:25.928531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.928510    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 16:20:25.928637 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.928595    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 16:20:25.928777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.928744    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lp92w\""
Jul 09 16:20:25.928852 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.928805    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 16:20:25.943245 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.943220    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9fbtj"]
Jul 09 16:20:25.943382 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.943332    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:25.945984 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.945962    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 16:20:25.946123 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.945985    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 16:20:25.946123 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.946078    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 16:20:25.946258 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.946240    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 16:20:25.946316 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:25.946306    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8pd94\""
Jul 09 16:20:26.073154 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073067    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-metrics-tls\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.073154 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073103    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7wsg\" (UniqueName: \"kubernetes.io/projected/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-kube-api-access-n7wsg\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.073154 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073124    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-crio-socket\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.073154 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073143    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.073446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073204    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-tmp-dir\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.073446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073259    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-477bc\" (UniqueName: \"kubernetes.io/projected/e68249db-71d5-47d9-b11a-d636f7967a37-kube-api-access-477bc\") pod \"ingress-canary-mgpzc\" (UID: \"e68249db-71d5-47d9-b11a-d636f7967a37\") " pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.073446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073299    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-config-volume\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.073446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073314    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e68249db-71d5-47d9-b11a-d636f7967a37-cert\") pod \"ingress-canary-mgpzc\" (UID: \"e68249db-71d5-47d9-b11a-d636f7967a37\") " pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.073446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073421    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-data-volume\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.073641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073460    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.073641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.073490    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxppt\" (UniqueName: \"kubernetes.io/projected/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-kube-api-access-cxppt\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.174631 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174602    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-config-volume\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.174781 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174642    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e68249db-71d5-47d9-b11a-d636f7967a37-cert\") pod \"ingress-canary-mgpzc\" (UID: \"e68249db-71d5-47d9-b11a-d636f7967a37\") " pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.174781 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174693    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-data-volume\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.174781 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174719    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.174945 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174813    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxppt\" (UniqueName: \"kubernetes.io/projected/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-kube-api-access-cxppt\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.174945 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174872    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-metrics-tls\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.174945 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174897    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7wsg\" (UniqueName: \"kubernetes.io/projected/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-kube-api-access-n7wsg\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.174945 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174924    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-crio-socket\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.175142 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174952    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.175142 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.174982    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-tmp-dir\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.175142 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.175021    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-477bc\" (UniqueName: \"kubernetes.io/projected/e68249db-71d5-47d9-b11a-d636f7967a37-kube-api-access-477bc\") pod \"ingress-canary-mgpzc\" (UID: \"e68249db-71d5-47d9-b11a-d636f7967a37\") " pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.175142 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.175058    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-data-volume\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.175339 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.175276    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-config-volume\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.175339 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.175299    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.175609 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.175576    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-tmp-dir\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.175609 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.175594    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-crio-socket\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.180181 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.179841    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-metrics-tls\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.180181 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.179842    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.180426 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.180389    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e68249db-71d5-47d9-b11a-d636f7967a37-cert\") pod \"ingress-canary-mgpzc\" (UID: \"e68249db-71d5-47d9-b11a-d636f7967a37\") " pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.183838 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.183793    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7wsg\" (UniqueName: \"kubernetes.io/projected/3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb-kube-api-access-n7wsg\") pod \"dns-default-nzgfq\" (UID: \"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb\") " pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.185342 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.185318    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxppt\" (UniqueName: \"kubernetes.io/projected/10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e-kube-api-access-cxppt\") pod \"insights-runtime-extractor-9fbtj\" (UID: \"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e\") " pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.185688 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.185659    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-477bc\" (UniqueName: \"kubernetes.io/projected/e68249db-71d5-47d9-b11a-d636f7967a37-kube-api-access-477bc\") pod \"ingress-canary-mgpzc\" (UID: \"e68249db-71d5-47d9-b11a-d636f7967a37\") " pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.226241 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.226196    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:26.233343 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.233320    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mgpzc"
Jul 09 16:20:26.252478 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.252259    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9fbtj"
Jul 09 16:20:26.413809 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.413779    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzgfq"]
Jul 09 16:20:26.421199 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:26.421158    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4fa0a2_b1f8_4f17_b864_6ff5f4079fbb.slice/crio-10012ea7e401976020089e5c0df356b837034cd5b2ff8116da87b7c7894b08e9 WatchSource:0}: Error finding container 10012ea7e401976020089e5c0df356b837034cd5b2ff8116da87b7c7894b08e9: Status 404 returned error can't find the container with id 10012ea7e401976020089e5c0df356b837034cd5b2ff8116da87b7c7894b08e9
Jul 09 16:20:26.547823 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.545958    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9fbtj"]
Jul 09 16:20:26.549890 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.549863    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mgpzc"]
Jul 09 16:20:26.552821 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:26.552791    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d53b6a_70d9_4eeb_a4cf_9b51e1000e9e.slice/crio-be3ebc2710b945c6777ce54be2011725d828d1ae827a51d9baf051f804487d06 WatchSource:0}: Error finding container be3ebc2710b945c6777ce54be2011725d828d1ae827a51d9baf051f804487d06: Status 404 returned error can't find the container with id be3ebc2710b945c6777ce54be2011725d828d1ae827a51d9baf051f804487d06
Jul 09 16:20:26.553865 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:26.553835    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68249db_71d5_47d9_b11a_d636f7967a37.slice/crio-ed318dfc85439ffc0ad1a1df543b3e554e855c749d8e0240625757754d3eefae WatchSource:0}: Error finding container ed318dfc85439ffc0ad1a1df543b3e554e855c749d8e0240625757754d3eefae: Status 404 returned error can't find the container with id ed318dfc85439ffc0ad1a1df543b3e554e855c749d8e0240625757754d3eefae
Jul 09 16:20:26.679411 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.679331    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f75f4cf87-nrcn5"]
Jul 09 16:20:26.711322 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.711285    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f75f4cf87-nrcn5"]
Jul 09 16:20:26.711474 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.711409    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.714306 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714281    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 16:20:26.714450 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714374    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 16:20:26.714542 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714525    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 16:20:26.714685 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714667    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 16:20:26.714825 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714693    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 16:20:26.714825 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714779    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tpmbb\""
Jul 09 16:20:26.714943 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714925    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 16:20:26.714999 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.714988    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 16:20:26.779406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.779372    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-service-ca\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.779877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.779423    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b5k8\" (UniqueName: \"kubernetes.io/projected/483f5f00-5aff-4ed0-b86d-a7a53b88814c-kube-api-access-7b5k8\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.779877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.779460    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-oauth-config\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.779877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.779486    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-oauth-serving-cert\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.779877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.779508    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-serving-cert\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.779877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.779523    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-config\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.880482 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.880451    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-service-ca\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.880482 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.880485    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b5k8\" (UniqueName: \"kubernetes.io/projected/483f5f00-5aff-4ed0-b86d-a7a53b88814c-kube-api-access-7b5k8\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.880709 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.880527    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-oauth-config\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.880709 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.880563    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-oauth-serving-cert\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.880709 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.880601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-serving-cert\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.880709 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.880625    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-config\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.881234 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.881207    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-service-ca\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.881346 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.881232    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-config\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.881346 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.881262    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-oauth-serving-cert\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.884159 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.884137    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-oauth-config\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.884159 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.884152    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-serving-cert\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.888660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.888640    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b5k8\" (UniqueName: \"kubernetes.io/projected/483f5f00-5aff-4ed0-b86d-a7a53b88814c-kube-api-access-7b5k8\") pod \"console-6f75f4cf87-nrcn5\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") " pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:26.960720 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.960634    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fbtj" event={"ID":"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e","Type":"ContainerStarted","Data":"be3ebc2710b945c6777ce54be2011725d828d1ae827a51d9baf051f804487d06"}
Jul 09 16:20:26.962983 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.962955    2586 generic.go:356] "Generic (PLEG): container finished" podID="48466a63-02d5-4325-a3c1-6e561a9a01b1" containerID="b0b8cabda02d778fde4e5b7e091a20a24b1b263594982758d24f0e9e4c01d091" exitCode=0
Jul 09 16:20:26.963115 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.962993    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerDied","Data":"b0b8cabda02d778fde4e5b7e091a20a24b1b263594982758d24f0e9e4c01d091"}
Jul 09 16:20:26.964371 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.964327    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mgpzc" event={"ID":"e68249db-71d5-47d9-b11a-d636f7967a37","Type":"ContainerStarted","Data":"ed318dfc85439ffc0ad1a1df543b3e554e855c749d8e0240625757754d3eefae"}
Jul 09 16:20:26.965683 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:26.965659    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzgfq" event={"ID":"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb","Type":"ContainerStarted","Data":"10012ea7e401976020089e5c0df356b837034cd5b2ff8116da87b7c7894b08e9"}
Jul 09 16:20:27.021024 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.020990    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:27.196809 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.196633    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f75f4cf87-nrcn5"]
Jul 09 16:20:27.768449 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.766556    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:27.768449 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.767028    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:27.768449 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.767436    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:27.770826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.769985    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 16:20:27.770826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.770296    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 16:20:27.771785 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.771246    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cg2dj\""
Jul 09 16:20:27.771785 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.771663    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 16:20:27.773124 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.772171    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 16:20:27.773124 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.772396    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q7g2m\""
Jul 09 16:20:27.971392 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:27.971318    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f75f4cf87-nrcn5" event={"ID":"483f5f00-5aff-4ed0-b86d-a7a53b88814c","Type":"ContainerStarted","Data":"746db3dd01e5984d04739ba5097df86f086902e5b5ce28422d401571d4bc0121"}
Jul 09 16:20:30.763715 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.763499    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-fkn6x"]
Jul 09 16:20:30.776079 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.776047    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:30.779745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.779231    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 16:20:30.779745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.779346    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 16:20:30.779745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.779482    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hc26v\""
Jul 09 16:20:30.779745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.779503    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 16:20:30.779745 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.779590    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 16:20:30.780160 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.779973    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 16:20:30.781227 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.781206    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-fkn6x"]
Jul 09 16:20:30.926932 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.926898    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:30.926932 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.926943    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:30.927157 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.926972    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g44kp\" (UniqueName: \"kubernetes.io/projected/cfed5735-fff9-4abd-b879-c20b6ac9bc14-kube-api-access-g44kp\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:30.927157 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:30.927100    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfed5735-fff9-4abd-b879-c20b6ac9bc14-metrics-client-ca\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.028410 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.028312    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.028410 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.028366    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44kp\" (UniqueName: \"kubernetes.io/projected/cfed5735-fff9-4abd-b879-c20b6ac9bc14-kube-api-access-g44kp\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.028660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.028433    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfed5735-fff9-4abd-b879-c20b6ac9bc14-metrics-client-ca\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.028660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.028515    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.028660 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:31.028603    2586 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 09 16:20:31.028660 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:31.028656    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls podName:cfed5735-fff9-4abd-b879-c20b6ac9bc14 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:31.528636877 +0000 UTC m=+42.454591136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls") pod "prometheus-operator-64785b6675-fkn6x" (UID: "cfed5735-fff9-4abd-b879-c20b6ac9bc14") : secret "prometheus-operator-tls" not found
Jul 09 16:20:31.029225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.029200    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cfed5735-fff9-4abd-b879-c20b6ac9bc14-metrics-client-ca\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.032046 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.032022    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.038842 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.038801    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g44kp\" (UniqueName: \"kubernetes.io/projected/cfed5735-fff9-4abd-b879-c20b6ac9bc14-kube-api-access-g44kp\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.531831 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:31.531801    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:31.532024 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:31.531957    2586 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 09 16:20:31.532080 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:31.532023    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls podName:cfed5735-fff9-4abd-b879-c20b6ac9bc14 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:32.532004053 +0000 UTC m=+43.457958311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls") pod "prometheus-operator-64785b6675-fkn6x" (UID: "cfed5735-fff9-4abd-b879-c20b6ac9bc14") : secret "prometheus-operator-tls" not found
Jul 09 16:20:32.539656 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:32.539621    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:32.542743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:32.542715    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/cfed5735-fff9-4abd-b879-c20b6ac9bc14-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-fkn6x\" (UID: \"cfed5735-fff9-4abd-b879-c20b6ac9bc14\") " pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:32.587951 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:32.587912    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x"
Jul 09 16:20:35.866178 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:35.866146    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-fkn6x"]
Jul 09 16:20:36.150107 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:36.150046    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfed5735_fff9_4abd_b879_c20b6ac9bc14.slice/crio-ee790bc6bc8f00855ef4de853267f84e2bb937be05aff4f00d41813001c9ae95 WatchSource:0}: Error finding container ee790bc6bc8f00855ef4de853267f84e2bb937be05aff4f00d41813001c9ae95: Status 404 returned error can't find the container with id ee790bc6bc8f00855ef4de853267f84e2bb937be05aff4f00d41813001c9ae95
Jul 09 16:20:36.996908 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:36.996868    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mgpzc" event={"ID":"e68249db-71d5-47d9-b11a-d636f7967a37","Type":"ContainerStarted","Data":"8a95b6d0557e27109c0c59cfbca81d115277b3ae0b03ff97a4154ae4b3b27c79"}
Jul 09 16:20:36.998365 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:36.998339    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fbtj" event={"ID":"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e","Type":"ContainerStarted","Data":"1f5db520f0d43c14f13a2204ee2d9d9bb89dc0b78f91bf50831a1d0584ce1ef2"}
Jul 09 16:20:36.999990 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:36.999961    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f75f4cf87-nrcn5" event={"ID":"483f5f00-5aff-4ed0-b86d-a7a53b88814c","Type":"ContainerStarted","Data":"851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e"}
Jul 09 16:20:37.002832 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.002808    2586 generic.go:356] "Generic (PLEG): container finished" podID="48466a63-02d5-4325-a3c1-6e561a9a01b1" containerID="610c7ec668ebaa10e81a896c1734df625a25746af9003f4863e45554693f0503" exitCode=0
Jul 09 16:20:37.002927 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.002888    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerDied","Data":"610c7ec668ebaa10e81a896c1734df625a25746af9003f4863e45554693f0503"}
Jul 09 16:20:37.005234 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.005019    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzgfq" event={"ID":"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb","Type":"ContainerStarted","Data":"0d0010769fe05a0bbc789be1ad35ee67d168d693c6b4cece6c07408b5cd93925"}
Jul 09 16:20:37.005234 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.005071    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:37.005234 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.005096    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzgfq" event={"ID":"3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb","Type":"ContainerStarted","Data":"89c3d8224f3f34b83969a68be02ba4218845c847217b192d30feb431292e8500"}
Jul 09 16:20:37.006331 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.006312    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x" event={"ID":"cfed5735-fff9-4abd-b879-c20b6ac9bc14","Type":"ContainerStarted","Data":"ee790bc6bc8f00855ef4de853267f84e2bb937be05aff4f00d41813001c9ae95"}
Jul 09 16:20:37.014359 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.014313    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mgpzc" podStartSLOduration=2.420834007 podStartE2EDuration="12.014294685s" totalImagesPullingTime="9.593460678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:37.013698667 +0000 UTC m=+47.939652946" watchObservedRunningTime="2026-07-09 16:20:37.014294685 +0000 UTC m=+47.940248965"
Jul 09 16:20:37.021399 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.021371    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:37.021539 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.021438    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:20:37.022803 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.022745    2586 patch_prober.go:28] interesting pod/console-6f75f4cf87-nrcn5 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 09 16:20:37.022885 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.022819    2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6f75f4cf87-nrcn5" podUID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 09 16:20:37.036239 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.036167    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nzgfq" podStartSLOduration=2.755262523 podStartE2EDuration="12.036145369s" totalImagesPullingTime="9.280882846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:37.035341283 +0000 UTC m=+47.961295594" watchObservedRunningTime="2026-07-09 16:20:37.036145369 +0000 UTC m=+47.962099647"
Jul 09 16:20:37.075130 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:37.075080    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f75f4cf87-nrcn5" podStartSLOduration=2.135894936 podStartE2EDuration="11.075054593s" totalImagesPullingTime="8.939159657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:37.074344876 +0000 UTC m=+48.000299156" watchObservedRunningTime="2026-07-09 16:20:37.075054593 +0000 UTC m=+48.001008852"
Jul 09 16:20:38.012110 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:38.012076    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fbtj" event={"ID":"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e","Type":"ContainerStarted","Data":"b99f310fff9534b5263bc16c16fd4a4f33ae44ba97f0eceb2c8954545f9338e0"}
Jul 09 16:20:38.014972 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:38.014947    2586 generic.go:356] "Generic (PLEG): container finished" podID="48466a63-02d5-4325-a3c1-6e561a9a01b1" containerID="3665af46760d10059b83e2768f89c0cac7d7dcb052d80644cc4d26f3ca97fe37" exitCode=0
Jul 09 16:20:38.015099 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:38.015022    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerDied","Data":"3665af46760d10059b83e2768f89c0cac7d7dcb052d80644cc4d26f3ca97fe37"}
Jul 09 16:20:38.016736 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:38.016714    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x" event={"ID":"cfed5735-fff9-4abd-b879-c20b6ac9bc14","Type":"ContainerStarted","Data":"5793e26b12115c0d4de2433d4e48a165e69a6bbe0419723d87386262d64b3288"}
Jul 09 16:20:39.026445 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:39.026402    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" event={"ID":"48466a63-02d5-4325-a3c1-6e561a9a01b1","Type":"ContainerStarted","Data":"32013c05c37638d5ec11adcb3ae59c1dec32eb3e0b0040c44cb6cb7931ccc71f"}
Jul 09 16:20:39.028731 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:39.028698    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x" event={"ID":"cfed5735-fff9-4abd-b879-c20b6ac9bc14","Type":"ContainerStarted","Data":"38605ad37dfac5643fc9a573e6c2da7285204043e873a53a3c57cb57365afdb2"}
Jul 09 16:20:39.051050 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:39.050991    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v5xwb" podStartSLOduration=13.690447719 podStartE2EDuration="48.05097187s" totalImagesPullingTime="34.360524151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:39.04920387 +0000 UTC m=+49.975158176" watchObservedRunningTime="2026-07-09 16:20:39.05097187 +0000 UTC m=+49.976926149"
Jul 09 16:20:39.068178 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:39.068104    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-fkn6x" podStartSLOduration=7.386627599 podStartE2EDuration="9.068082841s" totalImagesPullingTime="1.681455242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:39.06759999 +0000 UTC m=+49.993554269" watchObservedRunningTime="2026-07-09 16:20:39.068082841 +0000 UTC m=+49.994037139"
Jul 09 16:20:41.098378 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.098340    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"]
Jul 09 16:20:41.109001 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.108975    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.110915 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.110891    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"]
Jul 09 16:20:41.112301 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.112272    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 16:20:41.112471 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.112299    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fp8lh\""
Jul 09 16:20:41.112471 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.112426    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 16:20:41.114393 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.114372    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lqcnq"]
Jul 09 16:20:41.129569 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.129545    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.132362 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.132339    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 16:20:41.132652 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.132626    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 16:20:41.132843 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.132825    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 16:20:41.133093 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.132985    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tv8r2\""
Jul 09 16:20:41.214172 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.214136    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.214359 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.214201    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx94g\" (UniqueName: \"kubernetes.io/projected/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-kube-api-access-dx94g\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.214359 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.214223    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.214359 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.214240    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.314981 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.314887    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-wtmp\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.314981 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.314935    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-root\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.314981 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.314956    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqdr9\" (UniqueName: \"kubernetes.io/projected/7104417e-1224-4939-bca1-4a18b5ddc97e-kube-api-access-lqdr9\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.315192 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.314988    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-tls\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.315192 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315046    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7104417e-1224-4939-bca1-4a18b5ddc97e-metrics-client-ca\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.315192 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315128    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.315192 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315151    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.315192 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315180    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-accelerators-collector-config\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.315348 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315242    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.315348 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315266    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx94g\" (UniqueName: \"kubernetes.io/projected/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-kube-api-access-dx94g\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.315348 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315287    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-sys\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.315348 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315315    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.315348 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315338    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-textfile\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.316013 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.315989    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.317522 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.317504    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.317605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.317574    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.324057 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.324030    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx94g\" (UniqueName: \"kubernetes.io/projected/78a3efd9-412b-4cdb-bdf8-a4a1d965b129-kube-api-access-dx94g\") pod \"openshift-state-metrics-84bd68b896-n9mz7\" (UID: \"78a3efd9-412b-4cdb-bdf8-a4a1d965b129\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.416232 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416205    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416355 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416240    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-sys\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416355 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416262    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-textfile\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416355 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416332    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-sys\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416384    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-wtmp\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416433    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-root\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416450    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqdr9\" (UniqueName: \"kubernetes.io/projected/7104417e-1224-4939-bca1-4a18b5ddc97e-kube-api-access-lqdr9\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416478    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-root\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416483    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-tls\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416527    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7104417e-1224-4939-bca1-4a18b5ddc97e-metrics-client-ca\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416553    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-textfile\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416554    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-wtmp\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.416677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.416576    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-accelerators-collector-config\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.417242 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.417109    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7104417e-1224-4939-bca1-4a18b5ddc97e-metrics-client-ca\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.417242 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.417182    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-accelerators-collector-config\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.418680 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.418661    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-tls\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.418754 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.418736    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7104417e-1224-4939-bca1-4a18b5ddc97e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.418822 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.418744    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"
Jul 09 16:20:41.424585 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.424562    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqdr9\" (UniqueName: \"kubernetes.io/projected/7104417e-1224-4939-bca1-4a18b5ddc97e-kube-api-access-lqdr9\") pod \"node-exporter-lqcnq\" (UID: \"7104417e-1224-4939-bca1-4a18b5ddc97e\") " pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.441503 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.441480    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lqcnq"
Jul 09 16:20:41.464049 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:41.464021    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7104417e_1224_4939_bca1_4a18b5ddc97e.slice/crio-a047cfee4fe585e10d960b9fd3172a16b82ca2c2132222e7016a0b9e36af65ac WatchSource:0}: Error finding container a047cfee4fe585e10d960b9fd3172a16b82ca2c2132222e7016a0b9e36af65ac: Status 404 returned error can't find the container with id a047cfee4fe585e10d960b9fd3172a16b82ca2c2132222e7016a0b9e36af65ac
Jul 09 16:20:41.565396 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:41.565319    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7"]
Jul 09 16:20:41.568246 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:41.568220    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a3efd9_412b_4cdb_bdf8_a4a1d965b129.slice/crio-0770fcf51524c29fd96deaed2d08eada21f06154ad703d13aa701a1664c645ad WatchSource:0}: Error finding container 0770fcf51524c29fd96deaed2d08eada21f06154ad703d13aa701a1664c645ad: Status 404 returned error can't find the container with id 0770fcf51524c29fd96deaed2d08eada21f06154ad703d13aa701a1664c645ad
Jul 09 16:20:42.043380 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:42.043342    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9fbtj" event={"ID":"10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e","Type":"ContainerStarted","Data":"3e24f550b0df4ef4257f2626ff43b339be1d7114f855ecbf5fef1cb4b36a5a2c"}
Jul 09 16:20:42.044973 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:42.044935    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqcnq" event={"ID":"7104417e-1224-4939-bca1-4a18b5ddc97e","Type":"ContainerStarted","Data":"a047cfee4fe585e10d960b9fd3172a16b82ca2c2132222e7016a0b9e36af65ac"}
Jul 09 16:20:42.047029 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:42.047002    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7" event={"ID":"78a3efd9-412b-4cdb-bdf8-a4a1d965b129","Type":"ContainerStarted","Data":"bc58b320852b025727cc6b6a65fbd95eff2d6ef418db090822b942ab1c1c502c"}
Jul 09 16:20:42.047151 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:42.047058    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7" event={"ID":"78a3efd9-412b-4cdb-bdf8-a4a1d965b129","Type":"ContainerStarted","Data":"3840feaac218b2a2d1bf71a5d218fca27f959a7f1618356ab765cc7726f00e42"}
Jul 09 16:20:42.047151 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:42.047079    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7" event={"ID":"78a3efd9-412b-4cdb-bdf8-a4a1d965b129","Type":"ContainerStarted","Data":"0770fcf51524c29fd96deaed2d08eada21f06154ad703d13aa701a1664c645ad"}
Jul 09 16:20:42.069018 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:42.068679    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9fbtj" podStartSLOduration=2.840594806 podStartE2EDuration="17.068655841s" totalImagesPullingTime="14.228061035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:20:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:42.068576614 +0000 UTC m=+52.994530893" watchObservedRunningTime="2026-07-09 16:20:42.068655841 +0000 UTC m=+52.994610120"
Jul 09 16:20:43.052572 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:43.051648    2586 generic.go:356] "Generic (PLEG): container finished" podID="7104417e-1224-4939-bca1-4a18b5ddc97e" containerID="0ec01086c257de9fc4fdb1f5f69e6046d4bfe2c2b05ca6b38634f692a7130669" exitCode=0
Jul 09 16:20:43.053015 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:43.052608    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqcnq" event={"ID":"7104417e-1224-4939-bca1-4a18b5ddc97e","Type":"ContainerDied","Data":"0ec01086c257de9fc4fdb1f5f69e6046d4bfe2c2b05ca6b38634f692a7130669"}
Jul 09 16:20:44.057155 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.057121    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqcnq" event={"ID":"7104417e-1224-4939-bca1-4a18b5ddc97e","Type":"ContainerStarted","Data":"cc473f37ce5266c64e2c58cdfcab75d97f80ab4f5f9e0e5be2a218b89cf6696a"}
Jul 09 16:20:44.057601 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.057170    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lqcnq" event={"ID":"7104417e-1224-4939-bca1-4a18b5ddc97e","Type":"ContainerStarted","Data":"ceb9705f529c5e7bb541df9e17a0655a0664bfe64998f5f91c2202b1a3fc15fb"}
Jul 09 16:20:44.059100 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.059073    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7" event={"ID":"78a3efd9-412b-4cdb-bdf8-a4a1d965b129","Type":"ContainerStarted","Data":"e3cc1db93d748c51a42fdaaa1e14c37e0c3b1ef674255a763db971d195cce63d"}
Jul 09 16:20:44.078229 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.078170    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lqcnq" podStartSLOduration=1.9208043479999999 podStartE2EDuration="3.078154993s" totalImagesPullingTime="1.080287424s" totalInitContainerRuntime="77.063221ms" isStatefulPod=true podCreationTimestamp="2026-07-09 16:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:44.076877681 +0000 UTC m=+55.002831960" watchObservedRunningTime="2026-07-09 16:20:44.078154993 +0000 UTC m=+55.004109270"
Jul 09 16:20:44.101224 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.101170    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-n9mz7" podStartSLOduration=1.871562894 podStartE2EDuration="3.101154446s" totalImagesPullingTime="1.229591552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:44.100429595 +0000 UTC m=+55.026383874" watchObservedRunningTime="2026-07-09 16:20:44.101154446 +0000 UTC m=+55.027108718"
Jul 09 16:20:44.197253 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.197221    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-866b9cd557-z8tvs"]
Jul 09 16:20:44.199676 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.199658    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.203177 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.203156    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 16:20:44.205641 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.205616    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 16:20:44.206827 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.206809    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 16:20:44.206924 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.206811    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d49hoie8f0767\""
Jul 09 16:20:44.207083 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.207063    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 16:20:44.207178 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.207093    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 16:20:44.207900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.207886    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zk6xv\""
Jul 09 16:20:44.232041 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.232008    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-866b9cd557-z8tvs"]
Jul 09 16:20:44.238454 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238425    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-grpc-tls\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238467    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238494    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238553    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238585    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcdgn\" (UniqueName: \"kubernetes.io/projected/8120edc8-a045-4f1c-84fc-a18706e65419-kube-api-access-wcdgn\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238881 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238633    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8120edc8-a045-4f1c-84fc-a18706e65419-metrics-client-ca\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238881 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238698    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-tls\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.238881 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.238731    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339168 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339136    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339330 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339184    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339330 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339212    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcdgn\" (UniqueName: \"kubernetes.io/projected/8120edc8-a045-4f1c-84fc-a18706e65419-kube-api-access-wcdgn\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339330 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339232    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8120edc8-a045-4f1c-84fc-a18706e65419-metrics-client-ca\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339330 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339293    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-tls\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339500 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339435    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339554 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339495    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-grpc-tls\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.339554 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.339541    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.340202 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.340161    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8120edc8-a045-4f1c-84fc-a18706e65419-metrics-client-ca\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.342167 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.342140    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.342278 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.342259    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.342503 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.342481    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.342568 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.342486    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-grpc-tls\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.342568 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.342560    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-tls\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.342972 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.342954    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8120edc8-a045-4f1c-84fc-a18706e65419-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.350034 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.350013    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcdgn\" (UniqueName: \"kubernetes.io/projected/8120edc8-a045-4f1c-84fc-a18706e65419-kube-api-access-wcdgn\") pod \"thanos-querier-866b9cd557-z8tvs\" (UID: \"8120edc8-a045-4f1c-84fc-a18706e65419\") " pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.508448 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.508412    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:44.641130 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:44.641107    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-866b9cd557-z8tvs"]
Jul 09 16:20:44.643570 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:44.643544    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8120edc8_a045_4f1c_84fc_a18706e65419.slice/crio-3c51d3fbf136a40e679e5f38a9eeb15f086ed93540f9f6cf7ed5e816e6747448 WatchSource:0}: Error finding container 3c51d3fbf136a40e679e5f38a9eeb15f086ed93540f9f6cf7ed5e816e6747448: Status 404 returned error can't find the container with id 3c51d3fbf136a40e679e5f38a9eeb15f086ed93540f9f6cf7ed5e816e6747448
Jul 09 16:20:45.063012 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.062923    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"3c51d3fbf136a40e679e5f38a9eeb15f086ed93540f9f6cf7ed5e816e6747448"}
Jul 09 16:20:45.915243 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.915209    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"]
Jul 09 16:20:45.917136 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.917111    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:45.919427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.919406    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 09 16:20:45.919987 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.919960    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8nl7l\""
Jul 09 16:20:45.929047 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.929025    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"]
Jul 09 16:20:45.952221 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.952190    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01315eef-d12c-4064-9b36-b1e5426a6086-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:45.952380 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:45.952226    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wdk\" (UniqueName: \"kubernetes.io/projected/01315eef-d12c-4064-9b36-b1e5426a6086-kube-api-access-b5wdk\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.052986 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.052948    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01315eef-d12c-4064-9b36-b1e5426a6086-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.053145 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.052997    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wdk\" (UniqueName: \"kubernetes.io/projected/01315eef-d12c-4064-9b36-b1e5426a6086-kube-api-access-b5wdk\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.053145 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:46.053112    2586 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 09 16:20:46.053259 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:20:46.053194    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01315eef-d12c-4064-9b36-b1e5426a6086-monitoring-plugin-cert podName:01315eef-d12c-4064-9b36-b1e5426a6086 nodeName:}" failed. No retries permitted until 2026-07-09 16:20:46.553169687 +0000 UTC m=+57.479123947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/01315eef-d12c-4064-9b36-b1e5426a6086-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-d2rcf" (UID: "01315eef-d12c-4064-9b36-b1e5426a6086") : secret "monitoring-plugin-cert" not found
Jul 09 16:20:46.068599 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.068569    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wdk\" (UniqueName: \"kubernetes.io/projected/01315eef-d12c-4064-9b36-b1e5426a6086-kube-api-access-b5wdk\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.203085 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.203008    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f75f4cf87-nrcn5"]
Jul 09 16:20:46.557260 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.557166    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01315eef-d12c-4064-9b36-b1e5426a6086-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.559563 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.559530    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01315eef-d12c-4064-9b36-b1e5426a6086-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-d2rcf\" (UID: \"01315eef-d12c-4064-9b36-b1e5426a6086\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.827214 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.827144    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:46.961314 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:46.961285    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"]
Jul 09 16:20:47.019593 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.019566    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nzgfq"
Jul 09 16:20:47.282055 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.282022    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66f567c94f-cb7h9"]
Jul 09 16:20:47.284686 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.284664    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.294345 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.294319    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 16:20:47.313927 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.313892    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f567c94f-cb7h9"]
Jul 09 16:20:47.366026 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.365995    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-config\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.366026 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.366032    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-oauth-config\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.366299 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.366050    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-serving-cert\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.366299 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.366071    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-trusted-ca-bundle\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.366299 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.366133    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-oauth-serving-cert\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.366299 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.366267    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtp7\" (UniqueName: \"kubernetes.io/projected/6fd067ec-6b6a-447b-bd77-e8b133a02868-kube-api-access-dhtp7\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.366299 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.366292    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-service-ca\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.463884 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:47.463853    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01315eef_d12c_4064_9b36_b1e5426a6086.slice/crio-facc33f7ff6e47fcf28d98495f690961ddc77d161017f8921d5f1448f295a78a WatchSource:0}: Error finding container facc33f7ff6e47fcf28d98495f690961ddc77d161017f8921d5f1448f295a78a: Status 404 returned error can't find the container with id facc33f7ff6e47fcf28d98495f690961ddc77d161017f8921d5f1448f295a78a
Jul 09 16:20:47.466711 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466687    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-config\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.466897 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466722    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-oauth-config\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.466897 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466793    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-serving-cert\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.466897 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466820    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-trusted-ca-bundle\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.466897 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466843    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-oauth-serving-cert\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.467095 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466908    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtp7\" (UniqueName: \"kubernetes.io/projected/6fd067ec-6b6a-447b-bd77-e8b133a02868-kube-api-access-dhtp7\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.467095 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.466933    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-service-ca\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.467513 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.467464    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-config\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.469215 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.469166    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-trusted-ca-bundle\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.469315 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.469278    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-oauth-serving-cert\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.470524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.470475    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-oauth-config\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.470620 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.470533    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-service-ca\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.471681 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.471660    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-serving-cert\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.480136 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.480108    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtp7\" (UniqueName: \"kubernetes.io/projected/6fd067ec-6b6a-447b-bd77-e8b133a02868-kube-api-access-dhtp7\") pod \"console-66f567c94f-cb7h9\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") " pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.595254 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.595012    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:47.758344 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:47.758308    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66f567c94f-cb7h9"]
Jul 09 16:20:47.762166 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:47.762133    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd067ec_6b6a_447b_bd77_e8b133a02868.slice/crio-b754adc35f92db964446be13fcad448594dc03187420eb5eff63af28fe75ff83 WatchSource:0}: Error finding container b754adc35f92db964446be13fcad448594dc03187420eb5eff63af28fe75ff83: Status 404 returned error can't find the container with id b754adc35f92db964446be13fcad448594dc03187420eb5eff63af28fe75ff83
Jul 09 16:20:48.077270 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.077191    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf" event={"ID":"01315eef-d12c-4064-9b36-b1e5426a6086","Type":"ContainerStarted","Data":"facc33f7ff6e47fcf28d98495f690961ddc77d161017f8921d5f1448f295a78a"}
Jul 09 16:20:48.079618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.079591    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"0d9499ffcd9efc0913451b2ba36291952ca83be040c6a92a7aee3f1ac27834f9"}
Jul 09 16:20:48.079785 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.079667    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"02f65368b2fa3353e159f56860797eb658443f4e29c321653f9fe85b1e1d892c"}
Jul 09 16:20:48.079785 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.079710    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"be2462d2bc3b87d6a5eed2699de5c3ec028ac61362269e4ec1232410ceb7c867"}
Jul 09 16:20:48.081376 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.081322    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f567c94f-cb7h9" event={"ID":"6fd067ec-6b6a-447b-bd77-e8b133a02868","Type":"ContainerStarted","Data":"ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043"}
Jul 09 16:20:48.081376 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.081359    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f567c94f-cb7h9" event={"ID":"6fd067ec-6b6a-447b-bd77-e8b133a02868","Type":"ContainerStarted","Data":"b754adc35f92db964446be13fcad448594dc03187420eb5eff63af28fe75ff83"}
Jul 09 16:20:48.100387 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:48.100329    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66f567c94f-cb7h9" podStartSLOduration=1.100308042 podStartE2EDuration="1.100308042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:48.099503361 +0000 UTC m=+59.025457640" watchObservedRunningTime="2026-07-09 16:20:48.100308042 +0000 UTC m=+59.026262318"
Jul 09 16:20:50.090169 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.090129    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf" event={"ID":"01315eef-d12c-4064-9b36-b1e5426a6086","Type":"ContainerStarted","Data":"04e785a7b14c1f050fa8c5a47aa2fd7461c7cc7bb157f03f0d5aab6d3c448b25"}
Jul 09 16:20:50.090658 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.090305    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:50.093442 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.093407    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"0eafa5eab0edb3c810a02ee207a9fa3b391b0ea46847b94c385efd17af73dc9e"}
Jul 09 16:20:50.093593 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.093469    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"8181919f254d87e9d572571b725aae3ff5293f5633640b98e83cde608c486bc9"}
Jul 09 16:20:50.093593 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.093500    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" event={"ID":"8120edc8-a045-4f1c-84fc-a18706e65419","Type":"ContainerStarted","Data":"1065d1072d58f999811a922c724d6ea513f7847ab9c96ce72d5810206dfba52d"}
Jul 09 16:20:50.093593 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.093569    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:50.095443 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.095418    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf"
Jul 09 16:20:50.107092 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.107031    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-d2rcf" podStartSLOduration=3.503595951 podStartE2EDuration="5.107013424s" totalImagesPullingTime="1.603417473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:50.106555125 +0000 UTC m=+61.032509405" watchObservedRunningTime="2026-07-09 16:20:50.107013424 +0000 UTC m=+61.032967706"
Jul 09 16:20:50.131842 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:50.131795    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs" podStartSLOduration=1.927872796 podStartE2EDuration="6.131775696s" totalImagesPullingTime="4.2039029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:20:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:20:50.130697482 +0000 UTC m=+61.056651770" watchObservedRunningTime="2026-07-09 16:20:50.131775696 +0000 UTC m=+61.057729965"
Jul 09 16:20:51.954071 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:51.954042    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6qf5t"
Jul 09 16:20:56.103437 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.103407    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-866b9cd557-z8tvs"
Jul 09 16:20:56.553523 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.553412    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:56.553798 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.553534    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:56.556119 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.556093    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 16:20:56.556666 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.556648    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 16:20:56.567192 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.567162    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cbcc858-6c65-4abf-b5ca-62934d5952c3-metrics-certs\") pod \"network-metrics-daemon-5p8v6\" (UID: \"8cbcc858-6c65-4abf-b5ca-62934d5952c3\") " pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:56.567350 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.567190    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba0bf9e7-94ef-430a-915a-f80caa23e641-original-pull-secret\") pod \"global-pull-secret-syncer-fn7v8\" (UID: \"ba0bf9e7-94ef-430a-915a-f80caa23e641\") " pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:56.588053 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.588017    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q7g2m\""
Jul 09 16:20:56.595512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.595464    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5p8v6"
Jul 09 16:20:56.606631 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.606598    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fn7v8"
Jul 09 16:20:56.654815 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.654135    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:56.657240 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.657012    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 16:20:56.668456 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.668427    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 16:20:56.682673 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.682608    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl7wn\" (UniqueName: \"kubernetes.io/projected/97eb91c1-f07b-4165-81b5-586f3f38abeb-kube-api-access-kl7wn\") pod \"network-check-target-xtjdr\" (UID: \"97eb91c1-f07b-4165-81b5-586f3f38abeb\") " pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:56.762864 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.762646    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5p8v6"]
Jul 09 16:20:56.768727 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:56.768701    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbcc858_6c65_4abf_b5ca_62934d5952c3.slice/crio-2cde0e23a15a921944873945e61d7334d84f46106b78a4d6992f1ee1c4393e64 WatchSource:0}: Error finding container 2cde0e23a15a921944873945e61d7334d84f46106b78a4d6992f1ee1c4393e64: Status 404 returned error can't find the container with id 2cde0e23a15a921944873945e61d7334d84f46106b78a4d6992f1ee1c4393e64
Jul 09 16:20:56.781083 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.781056    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fn7v8"]
Jul 09 16:20:56.783648 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:56.783625    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba0bf9e7_94ef_430a_915a_f80caa23e641.slice/crio-7c54e1be80778e623a00d19e1811f0127bffba16b1b8c11a4153b07e4d0d4d6e WatchSource:0}: Error finding container 7c54e1be80778e623a00d19e1811f0127bffba16b1b8c11a4153b07e4d0d4d6e: Status 404 returned error can't find the container with id 7c54e1be80778e623a00d19e1811f0127bffba16b1b8c11a4153b07e4d0d4d6e
Jul 09 16:20:56.899400 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.899368    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cg2dj\""
Jul 09 16:20:56.907354 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:56.907318    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:20:57.035851 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.035816    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xtjdr"]
Jul 09 16:20:57.039023 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:20:57.038994    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97eb91c1_f07b_4165_81b5_586f3f38abeb.slice/crio-d49e32054ac43bcaa1229112892e3e1a3c66d97074ac2e1eeecb4c6802f1b3c6 WatchSource:0}: Error finding container d49e32054ac43bcaa1229112892e3e1a3c66d97074ac2e1eeecb4c6802f1b3c6: Status 404 returned error can't find the container with id d49e32054ac43bcaa1229112892e3e1a3c66d97074ac2e1eeecb4c6802f1b3c6
Jul 09 16:20:57.118003 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.117963    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xtjdr" event={"ID":"97eb91c1-f07b-4165-81b5-586f3f38abeb","Type":"ContainerStarted","Data":"d49e32054ac43bcaa1229112892e3e1a3c66d97074ac2e1eeecb4c6802f1b3c6"}
Jul 09 16:20:57.119092 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.119063    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5p8v6" event={"ID":"8cbcc858-6c65-4abf-b5ca-62934d5952c3","Type":"ContainerStarted","Data":"2cde0e23a15a921944873945e61d7334d84f46106b78a4d6992f1ee1c4393e64"}
Jul 09 16:20:57.120093 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.120070    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fn7v8" event={"ID":"ba0bf9e7-94ef-430a-915a-f80caa23e641","Type":"ContainerStarted","Data":"7c54e1be80778e623a00d19e1811f0127bffba16b1b8c11a4153b07e4d0d4d6e"}
Jul 09 16:20:57.597187 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.597150    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:57.597381 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.597222    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:57.602396 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:57.602369    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:58.129914 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:58.129886    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:20:59.131187 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:20:59.131157    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5p8v6" event={"ID":"8cbcc858-6c65-4abf-b5ca-62934d5952c3","Type":"ContainerStarted","Data":"d782d2499b89a2c648fdd94028be5120451a1b23d0dfa568b422793786454359"}
Jul 09 16:21:00.136264 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:00.136218    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5p8v6" event={"ID":"8cbcc858-6c65-4abf-b5ca-62934d5952c3","Type":"ContainerStarted","Data":"ed96ccbdd66f216d3baad8d89e7a064e444374ac4080f8c456ffc4b2ccfe5449"}
Jul 09 16:21:03.148743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:03.148705    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fn7v8" event={"ID":"ba0bf9e7-94ef-430a-915a-f80caa23e641","Type":"ContainerStarted","Data":"cfbcb614981c8a933f6712a0421a7d94b0c9539810652a8dca90a09df5e38848"}
Jul 09 16:21:03.150202 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:03.150172    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xtjdr" event={"ID":"97eb91c1-f07b-4165-81b5-586f3f38abeb","Type":"ContainerStarted","Data":"f5a77d0b17d124d0287df1c63cec003734acc5e1f48f54934a08497a5d46be42"}
Jul 09 16:21:03.150329 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:03.150315    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:21:03.164282 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:03.164232    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5p8v6" podStartSLOduration=69.98867581 podStartE2EDuration="1m12.1642144s" totalImagesPullingTime="2.17553859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:21:00.154093593 +0000 UTC m=+71.080047871" watchObservedRunningTime="2026-07-09 16:21:03.1642144 +0000 UTC m=+74.090168677"
Jul 09 16:21:03.164427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:03.164378    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fn7v8" podStartSLOduration=66.464223619 podStartE2EDuration="1m12.164373608s" totalImagesPullingTime="5.700149989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:21:03.163460489 +0000 UTC m=+74.089414766" watchObservedRunningTime="2026-07-09 16:21:03.164373608 +0000 UTC m=+74.090327919"
Jul 09 16:21:03.182819 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:03.182749    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xtjdr" podStartSLOduration=66.74949745 podStartE2EDuration="1m12.18273279s" totalImagesPullingTime="5.43323534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:21:03.181016052 +0000 UTC m=+74.106970329" watchObservedRunningTime="2026-07-09 16:21:03.18273279 +0000 UTC m=+74.108687068"
Jul 09 16:21:11.222477 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.222328    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f75f4cf87-nrcn5" podUID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" containerName="console" containerID="cri-o://851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e" gracePeriod=15
Jul 09 16:21:11.475635 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.475564    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:21:11.579839 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.579787    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-config\") pod \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") "
Jul 09 16:21:11.579839 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.579844    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-serving-cert\") pod \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") "
Jul 09 16:21:11.580049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.579889    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b5k8\" (UniqueName: \"kubernetes.io/projected/483f5f00-5aff-4ed0-b86d-a7a53b88814c-kube-api-access-7b5k8\") pod \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") "
Jul 09 16:21:11.580049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.579952    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-oauth-config\") pod \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") "
Jul 09 16:21:11.580049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.580005    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-service-ca\") pod \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") "
Jul 09 16:21:11.580049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.580035    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-oauth-serving-cert\") pod \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\" (UID: \"483f5f00-5aff-4ed0-b86d-a7a53b88814c\") "
Jul 09 16:21:11.580421 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.580394    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-config" (OuterVolumeSpecName: "console-config") pod "483f5f00-5aff-4ed0-b86d-a7a53b88814c" (UID: "483f5f00-5aff-4ed0-b86d-a7a53b88814c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:21:11.580486 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.580417    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-service-ca" (OuterVolumeSpecName: "service-ca") pod "483f5f00-5aff-4ed0-b86d-a7a53b88814c" (UID: "483f5f00-5aff-4ed0-b86d-a7a53b88814c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:21:11.580528 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.580498    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "483f5f00-5aff-4ed0-b86d-a7a53b88814c" (UID: "483f5f00-5aff-4ed0-b86d-a7a53b88814c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:21:11.582400 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.582372    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/483f5f00-5aff-4ed0-b86d-a7a53b88814c-kube-api-access-7b5k8" (OuterVolumeSpecName: "kube-api-access-7b5k8") pod "483f5f00-5aff-4ed0-b86d-a7a53b88814c" (UID: "483f5f00-5aff-4ed0-b86d-a7a53b88814c"). InnerVolumeSpecName "kube-api-access-7b5k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:21:11.582501 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.582376    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "483f5f00-5aff-4ed0-b86d-a7a53b88814c" (UID: "483f5f00-5aff-4ed0-b86d-a7a53b88814c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:21:11.582501 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.582428    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "483f5f00-5aff-4ed0-b86d-a7a53b88814c" (UID: "483f5f00-5aff-4ed0-b86d-a7a53b88814c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:21:11.680947 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.680895    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-oauth-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:21:11.680947 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.680937    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:21:11.680947 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.680947    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-oauth-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:21:11.680947 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.680956    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:21:11.680947 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.680965    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/483f5f00-5aff-4ed0-b86d-a7a53b88814c-console-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:21:11.681245 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:11.680973    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b5k8\" (UniqueName: \"kubernetes.io/projected/483f5f00-5aff-4ed0-b86d-a7a53b88814c-kube-api-access-7b5k8\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:21:12.192071 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.192032    2586 generic.go:356] "Generic (PLEG): container finished" podID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" containerID="851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e" exitCode=0
Jul 09 16:21:12.192232 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.192080    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f75f4cf87-nrcn5"
Jul 09 16:21:12.192232 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.192090    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f75f4cf87-nrcn5" event={"ID":"483f5f00-5aff-4ed0-b86d-a7a53b88814c","Type":"ContainerDied","Data":"851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e"}
Jul 09 16:21:12.192232 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.192143    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f75f4cf87-nrcn5" event={"ID":"483f5f00-5aff-4ed0-b86d-a7a53b88814c","Type":"ContainerDied","Data":"746db3dd01e5984d04739ba5097df86f086902e5b5ce28422d401571d4bc0121"}
Jul 09 16:21:12.192232 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.192171    2586 scope.go:117] "RemoveContainer" containerID="851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e"
Jul 09 16:21:12.201382 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.201359    2586 scope.go:117] "RemoveContainer" containerID="851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e"
Jul 09 16:21:12.201663 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:21:12.201644    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e\": container with ID starting with 851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e not found: ID does not exist" containerID="851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e"
Jul 09 16:21:12.201715 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.201673    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e"} err="failed to get container status \"851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e\": rpc error: code = NotFound desc = could not find container \"851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e\": container with ID starting with 851cd25fea09f35bd40a5584d7f9ffee8d8c2a87d387725a8f8812804854e72e not found: ID does not exist"
Jul 09 16:21:12.212784 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.212732    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f75f4cf87-nrcn5"]
Jul 09 16:21:12.217874 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.217835    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f75f4cf87-nrcn5"]
Jul 09 16:21:12.771820 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:12.771787    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" path="/var/lib/kubelet/pods/483f5f00-5aff-4ed0-b86d-a7a53b88814c/volumes"
Jul 09 16:21:34.157922 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:21:34.157890    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xtjdr"
Jul 09 16:22:03.473713 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.473678    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 16:22:03.474195 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.473994    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" containerName="console"
Jul 09 16:22:03.474195 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.474007    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" containerName="console"
Jul 09 16:22:03.474195 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.474059    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="483f5f00-5aff-4ed0-b86d-a7a53b88814c" containerName="console"
Jul 09 16:22:03.477650 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.477627    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.480544 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480508    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 16:22:03.480544 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480509    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 16:22:03.480734 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480510    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 16:22:03.480734 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480535    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 16:22:03.480734 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480542    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 16:22:03.480917 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480902    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 16:22:03.480972 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480945    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 16:22:03.481024 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480990    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gcf45\""
Jul 09 16:22:03.481074 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.480992    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 16:22:03.487798 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.487779    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 16:22:03.503936 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.503904    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 16:22:03.577206 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577168    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6989644a-4bf7-4977-8978-274e296c0db0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577250    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577292    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-web-config\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577326    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577357    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577400    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577432    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6989644a-4bf7-4977-8978-274e296c0db0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577463    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6989644a-4bf7-4977-8978-274e296c0db0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577500    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577530    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-config-volume\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577560    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6989644a-4bf7-4977-8978-274e296c0db0-config-out\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577594    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6989644a-4bf7-4977-8978-274e296c0db0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.577645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.577633    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcnvd\" (UniqueName: \"kubernetes.io/projected/6989644a-4bf7-4977-8978-274e296c0db0-kube-api-access-rcnvd\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678307 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678271    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6989644a-4bf7-4977-8978-274e296c0db0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678307 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678311    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcnvd\" (UniqueName: \"kubernetes.io/projected/6989644a-4bf7-4977-8978-274e296c0db0-kube-api-access-rcnvd\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678339    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6989644a-4bf7-4977-8978-274e296c0db0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678364    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678386    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-web-config\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678402    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678428    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678449    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678487    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6989644a-4bf7-4977-8978-274e296c0db0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678511    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6989644a-4bf7-4977-8978-274e296c0db0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678539    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.678584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678568    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-config-volume\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.679052 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678593    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6989644a-4bf7-4977-8978-274e296c0db0-config-out\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.679052 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.678791    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6989644a-4bf7-4977-8978-274e296c0db0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.679832 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.679343    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6989644a-4bf7-4977-8978-274e296c0db0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.679832 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.679555    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6989644a-4bf7-4977-8978-274e296c0db0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.681705 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.681668    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.681826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.681747    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.682036 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.681992    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6989644a-4bf7-4977-8978-274e296c0db0-config-out\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.682141 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.682100    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-web-config\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.682190 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.682174    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.682266 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.682244    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6989644a-4bf7-4977-8978-274e296c0db0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.682304 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.682258    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.682545 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.682527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-config-volume\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.683227 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.683208    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6989644a-4bf7-4977-8978-274e296c0db0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.694051 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.694024    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcnvd\" (UniqueName: \"kubernetes.io/projected/6989644a-4bf7-4977-8978-274e296c0db0-kube-api-access-rcnvd\") pod \"alertmanager-main-0\" (UID: \"6989644a-4bf7-4977-8978-274e296c0db0\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.787996 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.787897    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 16:22:03.931443 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:03.931414    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 16:22:03.932949 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:22:03.932925    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6989644a_4bf7_4977_8978_274e296c0db0.slice/crio-f007e3dff378b0bc19169f0d5c66cb390ae7bd41db60045342fb436071100467 WatchSource:0}: Error finding container f007e3dff378b0bc19169f0d5c66cb390ae7bd41db60045342fb436071100467: Status 404 returned error can't find the container with id f007e3dff378b0bc19169f0d5c66cb390ae7bd41db60045342fb436071100467
Jul 09 16:22:04.379220 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:04.379181    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"f007e3dff378b0bc19169f0d5c66cb390ae7bd41db60045342fb436071100467"}
Jul 09 16:22:05.384669 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:05.384632    2586 generic.go:356] "Generic (PLEG): container finished" podID="6989644a-4bf7-4977-8978-274e296c0db0" containerID="7b09fa8b7215bf9be0926d8b35cd723fcdd6553f910821310ef46bd93255b316" exitCode=0
Jul 09 16:22:05.384669 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:05.384672    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerDied","Data":"7b09fa8b7215bf9be0926d8b35cd723fcdd6553f910821310ef46bd93255b316"}
Jul 09 16:22:07.395182 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.395145    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"de906c042234ef1bf3b650f8c0c911f5ebdc7fb68d67b7d9657f8a48c2c8ff20"}
Jul 09 16:22:07.395566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.395215    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"2dc064f0a0215243a1493e75fdd8396d686fde83eec144f1f96d25b4e05d384b"}
Jul 09 16:22:07.395566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.395246    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"120ecd778e5e7727ce3252533bed3fa2ab58826add6c56e46e8dbcacbe507b86"}
Jul 09 16:22:07.395566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.395271    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"a1951ef0870d1eb73061df33f4ede9fbc88c19c4ac8dd6ea08ed7c95d1c425f1"}
Jul 09 16:22:07.395566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.395303    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"693906ca534fbb3cfbeba48c8ee5548491d6ae0e3eefa071f13422567de7f265"}
Jul 09 16:22:07.395566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.395330    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6989644a-4bf7-4977-8978-274e296c0db0","Type":"ContainerStarted","Data":"98de53feaf39b288313d7925e827279949a924e36283af7aa4fa344e2e4dcb57"}
Jul 09 16:22:07.427895 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:07.427837    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.8908462560000001 podStartE2EDuration="4.427813579s" totalImagesPullingTime="2.517734574s" totalInitContainerRuntime="19.232749ms" isStatefulPod=false podCreationTimestamp="2026-07-09 16:22:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:22:07.427412882 +0000 UTC m=+138.353367161" watchObservedRunningTime="2026-07-09 16:22:07.427813579 +0000 UTC m=+138.353767858"
Jul 09 16:22:10.670776 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.670728    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7d7b5698-qw5ml"]
Jul 09 16:22:10.672839 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.672817    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.691849 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.691827    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7d7b5698-qw5ml"]
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742167    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-trusted-ca-bundle\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742216    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-service-ca\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742251    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-config\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742315    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-oauth-serving-cert\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742341    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-oauth-config\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742366    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-serving-cert\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.742512 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.742397    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnsc4\" (UniqueName: \"kubernetes.io/projected/4a3520a9-83df-49a1-a176-e17e0e2f590d-kube-api-access-xnsc4\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843043 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.842998    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-oauth-serving-cert\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843043 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843036    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-oauth-config\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843043 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843055    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-serving-cert\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843351 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843178    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnsc4\" (UniqueName: \"kubernetes.io/projected/4a3520a9-83df-49a1-a176-e17e0e2f590d-kube-api-access-xnsc4\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843351 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843248    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-trusted-ca-bundle\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843351 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843285    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-service-ca\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843351 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843318    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-config\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.843926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.843898    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-oauth-serving-cert\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.844093 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.844065    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-trusted-ca-bundle\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.844258 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.844231    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-service-ca\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.844469 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.844443    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-config\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.846009 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.845965    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-serving-cert\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.847492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.847473    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-oauth-config\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.853433 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.853410    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnsc4\" (UniqueName: \"kubernetes.io/projected/4a3520a9-83df-49a1-a176-e17e0e2f590d-kube-api-access-xnsc4\") pod \"console-6b7d7b5698-qw5ml\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") " pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:10.984249 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:10.984148    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:11.118543 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:11.118518    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7d7b5698-qw5ml"]
Jul 09 16:22:11.121016 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:22:11.120985    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3520a9_83df_49a1_a176_e17e0e2f590d.slice/crio-cdc413c0879b6b060c9df7b396d8e18f8db9463a57b0c81dbb1adcff11ca18fd WatchSource:0}: Error finding container cdc413c0879b6b060c9df7b396d8e18f8db9463a57b0c81dbb1adcff11ca18fd: Status 404 returned error can't find the container with id cdc413c0879b6b060c9df7b396d8e18f8db9463a57b0c81dbb1adcff11ca18fd
Jul 09 16:22:11.409721 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:11.409685    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7d7b5698-qw5ml" event={"ID":"4a3520a9-83df-49a1-a176-e17e0e2f590d","Type":"ContainerStarted","Data":"ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05"}
Jul 09 16:22:11.409905 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:11.409741    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7d7b5698-qw5ml" event={"ID":"4a3520a9-83df-49a1-a176-e17e0e2f590d","Type":"ContainerStarted","Data":"cdc413c0879b6b060c9df7b396d8e18f8db9463a57b0c81dbb1adcff11ca18fd"}
Jul 09 16:22:11.433085 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:11.433026    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7d7b5698-qw5ml" podStartSLOduration=1.433006457 podStartE2EDuration="1.433006457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:22:11.432776305 +0000 UTC m=+142.358730576" watchObservedRunningTime="2026-07-09 16:22:11.433006457 +0000 UTC m=+142.358960735"
Jul 09 16:22:20.984440 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:20.984399    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:20.984962 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:20.984470    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:20.989227 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:20.989205    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:21.450615 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:21.450590    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:22:21.539160 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:21.539129    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f567c94f-cb7h9"]
Jul 09 16:22:46.558626 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.558541    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66f567c94f-cb7h9" podUID="6fd067ec-6b6a-447b-bd77-e8b133a02868" containerName="console" containerID="cri-o://ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043" gracePeriod=15
Jul 09 16:22:46.827319 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.827295    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:22:46.838262 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838238    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-oauth-config\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.838381 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838290    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-serving-cert\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.838381 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838314    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-service-ca\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.838381 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838346    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhtp7\" (UniqueName: \"kubernetes.io/projected/6fd067ec-6b6a-447b-bd77-e8b133a02868-kube-api-access-dhtp7\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.838542 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838384    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-config\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.838746 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838708    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-service-ca" (OuterVolumeSpecName: "service-ca") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:22:46.838889 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.838811    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-config" (OuterVolumeSpecName: "console-config") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:22:46.840493 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.840468    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:22:46.840608 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.840569    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:22:46.840608 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.840582    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd067ec-6b6a-447b-bd77-e8b133a02868-kube-api-access-dhtp7" (OuterVolumeSpecName: "kube-api-access-dhtp7") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "kube-api-access-dhtp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:22:46.938989 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.938951    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-trusted-ca-bundle\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.938989 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.938988    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-oauth-serving-cert\") pod \"6fd067ec-6b6a-447b-bd77-e8b133a02868\" (UID: \"6fd067ec-6b6a-447b-bd77-e8b133a02868\") "
Jul 09 16:22:46.939219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939203    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-oauth-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:46.939256 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939226    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:46.939256 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939240    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:46.939316 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939254    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhtp7\" (UniqueName: \"kubernetes.io/projected/6fd067ec-6b6a-447b-bd77-e8b133a02868-kube-api-access-dhtp7\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:46.939316 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939270    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-console-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:46.939391 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939374    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:22:46.939426 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:46.939390    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6fd067ec-6b6a-447b-bd77-e8b133a02868" (UID: "6fd067ec-6b6a-447b-bd77-e8b133a02868"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:22:47.039777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.039720    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-trusted-ca-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:47.039777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.039748    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd067ec-6b6a-447b-bd77-e8b133a02868-oauth-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:22:47.543296 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.543263    2586 generic.go:356] "Generic (PLEG): container finished" podID="6fd067ec-6b6a-447b-bd77-e8b133a02868" containerID="ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043" exitCode=0
Jul 09 16:22:47.543497 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.543308    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66f567c94f-cb7h9"
Jul 09 16:22:47.543497 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.543332    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f567c94f-cb7h9" event={"ID":"6fd067ec-6b6a-447b-bd77-e8b133a02868","Type":"ContainerDied","Data":"ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043"}
Jul 09 16:22:47.543497 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.543378    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66f567c94f-cb7h9" event={"ID":"6fd067ec-6b6a-447b-bd77-e8b133a02868","Type":"ContainerDied","Data":"b754adc35f92db964446be13fcad448594dc03187420eb5eff63af28fe75ff83"}
Jul 09 16:22:47.543497 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.543403    2586 scope.go:117] "RemoveContainer" containerID="ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043"
Jul 09 16:22:47.551775 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.551736    2586 scope.go:117] "RemoveContainer" containerID="ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043"
Jul 09 16:22:47.552071 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:22:47.552053    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043\": container with ID starting with ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043 not found: ID does not exist" containerID="ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043"
Jul 09 16:22:47.552115 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.552081    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043"} err="failed to get container status \"ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043\": rpc error: code = NotFound desc = could not find container \"ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043\": container with ID starting with ed5f0fe2d3c1b05e976ba88279ddd70b4bfddeeca29379f137831841b06bb043 not found: ID does not exist"
Jul 09 16:22:47.570778 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.570741    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66f567c94f-cb7h9"]
Jul 09 16:22:47.579801 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:47.579775    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66f567c94f-cb7h9"]
Jul 09 16:22:48.771651 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:22:48.771620    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd067ec-6b6a-447b-bd77-e8b133a02868" path="/var/lib/kubelet/pods/6fd067ec-6b6a-447b-bd77-e8b133a02868/volumes"
Jul 09 16:23:16.989238 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:16.989198    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75d5454df9-tsm28"]
Jul 09 16:23:16.989630 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:16.989537    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd067ec-6b6a-447b-bd77-e8b133a02868" containerName="console"
Jul 09 16:23:16.989630 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:16.989552    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd067ec-6b6a-447b-bd77-e8b133a02868" containerName="console"
Jul 09 16:23:16.989630 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:16.989603    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd067ec-6b6a-447b-bd77-e8b133a02868" containerName="console"
Jul 09 16:23:16.992426 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:16.992408    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.008036 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.008007    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75d5454df9-tsm28"]
Jul 09 16:23:17.056920 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.056874    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-oauth-serving-cert\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.056920 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.056925    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gw8s\" (UniqueName: \"kubernetes.io/projected/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-kube-api-access-6gw8s\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.057126 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.057008    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-config\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.057126 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.057070    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-serving-cert\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.057126 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.057106    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-oauth-config\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.057247 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.057132    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-service-ca\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.057247 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.057155    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-trusted-ca-bundle\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158371 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158332    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-config\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158393    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-serving-cert\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158435    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-oauth-config\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158465    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-service-ca\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158488    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-trusted-ca-bundle\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158521    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-oauth-serving-cert\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.158551 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.158547    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gw8s\" (UniqueName: \"kubernetes.io/projected/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-kube-api-access-6gw8s\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.159240 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.159210    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-config\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.159316 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.159258    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-service-ca\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.159316 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.159279    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-oauth-serving-cert\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.159380 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.159336    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-trusted-ca-bundle\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.160913 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.160892    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-oauth-config\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.161105 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.161089    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-serving-cert\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.167965 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.167944    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gw8s\" (UniqueName: \"kubernetes.io/projected/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-kube-api-access-6gw8s\") pod \"console-75d5454df9-tsm28\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") " pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.301808 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.301705    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:17.432437 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.432411    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75d5454df9-tsm28"]
Jul 09 16:23:17.434823 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:23:17.434797    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad4ddc2_a39a_46c8_a3ab_3bf7f04e25fc.slice/crio-189aaf3d03531831c5bb6de490dddcb45e76c0b3ee647aa0c11786989afbac9b WatchSource:0}: Error finding container 189aaf3d03531831c5bb6de490dddcb45e76c0b3ee647aa0c11786989afbac9b: Status 404 returned error can't find the container with id 189aaf3d03531831c5bb6de490dddcb45e76c0b3ee647aa0c11786989afbac9b
Jul 09 16:23:17.652835 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.652796    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d5454df9-tsm28" event={"ID":"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc","Type":"ContainerStarted","Data":"080c95c696e21a18a09c282e0ed966bfee04d3f911a23bb4f7faf56ca2f53a8b"}
Jul 09 16:23:17.653019 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.652857    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d5454df9-tsm28" event={"ID":"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc","Type":"ContainerStarted","Data":"189aaf3d03531831c5bb6de490dddcb45e76c0b3ee647aa0c11786989afbac9b"}
Jul 09 16:23:17.674863 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:17.674815    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75d5454df9-tsm28" podStartSLOduration=1.674800126 podStartE2EDuration="1.674800126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:23:17.672688005 +0000 UTC m=+208.598642283" watchObservedRunningTime="2026-07-09 16:23:17.674800126 +0000 UTC m=+208.600754404"
Jul 09 16:23:27.302101 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:27.302051    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:27.302101 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:27.302117    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:27.306981 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:27.306960    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:27.693833 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:27.693805    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:23:27.756474 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:27.756444    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7d7b5698-qw5ml"]
Jul 09 16:23:52.775975 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:52.775939    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b7d7b5698-qw5ml" podUID="4a3520a9-83df-49a1-a176-e17e0e2f590d" containerName="console" containerID="cri-o://ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05" gracePeriod=15
Jul 09 16:23:53.018170 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.018133    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:23:53.055053 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.054975    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnsc4\" (UniqueName: \"kubernetes.io/projected/4a3520a9-83df-49a1-a176-e17e0e2f590d-kube-api-access-xnsc4\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055053 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055020    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-serving-cert\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055053 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055040    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-config\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055264 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055223    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-service-ca\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055353 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055330    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-oauth-serving-cert\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055410 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055366    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-oauth-config\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055410 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055392    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-trusted-ca-bundle\") pod \"4a3520a9-83df-49a1-a176-e17e0e2f590d\" (UID: \"4a3520a9-83df-49a1-a176-e17e0e2f590d\") "
Jul 09 16:23:53.055514 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055465    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-config" (OuterVolumeSpecName: "console-config") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:23:53.055584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055535    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:23:53.055676 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055658    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.055676 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055677    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.055839 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055691    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:23:53.055918 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.055896    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:23:53.057132 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.057110    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:23:53.057266 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.057246    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3520a9-83df-49a1-a176-e17e0e2f590d-kube-api-access-xnsc4" (OuterVolumeSpecName: "kube-api-access-xnsc4") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "kube-api-access-xnsc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:23:53.057312 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.057283    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a3520a9-83df-49a1-a176-e17e0e2f590d" (UID: "4a3520a9-83df-49a1-a176-e17e0e2f590d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:23:53.156605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.156574    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnsc4\" (UniqueName: \"kubernetes.io/projected/4a3520a9-83df-49a1-a176-e17e0e2f590d-kube-api-access-xnsc4\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.156605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.156600    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.156605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.156611    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-oauth-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.156848 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.156619    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a3520a9-83df-49a1-a176-e17e0e2f590d-console-oauth-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.156848 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.156627    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a3520a9-83df-49a1-a176-e17e0e2f590d-trusted-ca-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:23:53.788742 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.788710    2586 generic.go:356] "Generic (PLEG): container finished" podID="4a3520a9-83df-49a1-a176-e17e0e2f590d" containerID="ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05" exitCode=0
Jul 09 16:23:53.789191 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.788752    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7d7b5698-qw5ml"
Jul 09 16:23:53.789191 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.788788    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7d7b5698-qw5ml" event={"ID":"4a3520a9-83df-49a1-a176-e17e0e2f590d","Type":"ContainerDied","Data":"ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05"}
Jul 09 16:23:53.789191 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.788832    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7d7b5698-qw5ml" event={"ID":"4a3520a9-83df-49a1-a176-e17e0e2f590d","Type":"ContainerDied","Data":"cdc413c0879b6b060c9df7b396d8e18f8db9463a57b0c81dbb1adcff11ca18fd"}
Jul 09 16:23:53.789191 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.788851    2586 scope.go:117] "RemoveContainer" containerID="ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05"
Jul 09 16:23:53.797144 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.797127    2586 scope.go:117] "RemoveContainer" containerID="ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05"
Jul 09 16:23:53.797403 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:23:53.797376    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05\": container with ID starting with ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05 not found: ID does not exist" containerID="ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05"
Jul 09 16:23:53.797468 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.797417    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05"} err="failed to get container status \"ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05\": rpc error: code = NotFound desc = could not find container \"ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05\": container with ID starting with ac627fdce0737db56216051419b5279260af9205e7e7beeae790e72c19224e05 not found: ID does not exist"
Jul 09 16:23:53.809666 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.809645    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7d7b5698-qw5ml"]
Jul 09 16:23:53.815599 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:53.815579    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7d7b5698-qw5ml"]
Jul 09 16:23:54.775440 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:23:54.775388    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a3520a9-83df-49a1-a176-e17e0e2f590d" path="/var/lib/kubelet/pods/4a3520a9-83df-49a1-a176-e17e0e2f590d/volumes"
Jul 09 16:24:50.601053 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:50.601026    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:24:50.602876 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:50.602849    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:24:50.604647 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:50.604626    2586 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 16:24:57.180864 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.180834    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"]
Jul 09 16:24:57.183106 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.181112    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3520a9-83df-49a1-a176-e17e0e2f590d" containerName="console"
Jul 09 16:24:57.183106 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.181124    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3520a9-83df-49a1-a176-e17e0e2f590d" containerName="console"
Jul 09 16:24:57.183106 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.181204    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3520a9-83df-49a1-a176-e17e0e2f590d" containerName="console"
Jul 09 16:24:57.183961 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.183944    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.186534 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.186513    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 16:24:57.186694 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.186675    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 16:24:57.186802 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.186786    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n5v5x\""
Jul 09 16:24:57.194171 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.194151    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"]
Jul 09 16:24:57.343999 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.343961    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4szr\" (UniqueName: \"kubernetes.io/projected/e24a9dff-fdaf-4960-8022-016578cfd31f-kube-api-access-t4szr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.344184 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.344027    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.344184 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.344080    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.445498 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.445414    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4szr\" (UniqueName: \"kubernetes.io/projected/e24a9dff-fdaf-4960-8022-016578cfd31f-kube-api-access-t4szr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.445498 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.445488    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.445683 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.445542    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.445912 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.445889    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.445956 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.445921    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.454529 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.454503    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4szr\" (UniqueName: \"kubernetes.io/projected/e24a9dff-fdaf-4960-8022-016578cfd31f-kube-api-access-t4szr\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.492520 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.492479    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:24:57.617565 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.617535    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"]
Jul 09 16:24:57.620855 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:24:57.620827    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24a9dff_fdaf_4960_8022_016578cfd31f.slice/crio-29da0d95a76bef3be06fcde365b9d608c98390868156279368325b06ba53e969 WatchSource:0}: Error finding container 29da0d95a76bef3be06fcde365b9d608c98390868156279368325b06ba53e969: Status 404 returned error can't find the container with id 29da0d95a76bef3be06fcde365b9d608c98390868156279368325b06ba53e969
Jul 09 16:24:57.622857 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:57.622839    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 16:24:58.022844 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:24:58.022813    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6" event={"ID":"e24a9dff-fdaf-4960-8022-016578cfd31f","Type":"ContainerStarted","Data":"29da0d95a76bef3be06fcde365b9d608c98390868156279368325b06ba53e969"}
Jul 09 16:25:03.046600 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:03.046512    2586 generic.go:356] "Generic (PLEG): container finished" podID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerID="53080ee7751016e7400a7018798a49d9e2866aa3119f3c0aa13592b502113c5a" exitCode=0
Jul 09 16:25:03.046979 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:03.046596    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6" event={"ID":"e24a9dff-fdaf-4960-8022-016578cfd31f","Type":"ContainerDied","Data":"53080ee7751016e7400a7018798a49d9e2866aa3119f3c0aa13592b502113c5a"}
Jul 09 16:25:06.059376 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:06.059342    2586 generic.go:356] "Generic (PLEG): container finished" podID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerID="bee613b294d12200bdc7b9606aba4ca27f7ed11d1e514bbfb50820a14ed2a8a2" exitCode=0
Jul 09 16:25:06.059894 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:06.059383    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6" event={"ID":"e24a9dff-fdaf-4960-8022-016578cfd31f","Type":"ContainerDied","Data":"bee613b294d12200bdc7b9606aba4ca27f7ed11d1e514bbfb50820a14ed2a8a2"}
Jul 09 16:25:12.087173 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:12.087140    2586 generic.go:356] "Generic (PLEG): container finished" podID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerID="f6724109774faee77bcf2eb96bf3964a0b0a43687f0f6ee248f57a29d3c96bde" exitCode=0
Jul 09 16:25:12.087546 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:12.087217    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6" event={"ID":"e24a9dff-fdaf-4960-8022-016578cfd31f","Type":"ContainerDied","Data":"f6724109774faee77bcf2eb96bf3964a0b0a43687f0f6ee248f57a29d3c96bde"}
Jul 09 16:25:13.219129 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.219101    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:25:13.384252 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.384165    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-util\") pod \"e24a9dff-fdaf-4960-8022-016578cfd31f\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") "
Jul 09 16:25:13.384252 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.384247    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-bundle\") pod \"e24a9dff-fdaf-4960-8022-016578cfd31f\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") "
Jul 09 16:25:13.384441 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.384280    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4szr\" (UniqueName: \"kubernetes.io/projected/e24a9dff-fdaf-4960-8022-016578cfd31f-kube-api-access-t4szr\") pod \"e24a9dff-fdaf-4960-8022-016578cfd31f\" (UID: \"e24a9dff-fdaf-4960-8022-016578cfd31f\") "
Jul 09 16:25:13.384907 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.384881    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-bundle" (OuterVolumeSpecName: "bundle") pod "e24a9dff-fdaf-4960-8022-016578cfd31f" (UID: "e24a9dff-fdaf-4960-8022-016578cfd31f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:25:13.386515 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.386492    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24a9dff-fdaf-4960-8022-016578cfd31f-kube-api-access-t4szr" (OuterVolumeSpecName: "kube-api-access-t4szr") pod "e24a9dff-fdaf-4960-8022-016578cfd31f" (UID: "e24a9dff-fdaf-4960-8022-016578cfd31f"). InnerVolumeSpecName "kube-api-access-t4szr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:25:13.388858 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.388836    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-util" (OuterVolumeSpecName: "util") pod "e24a9dff-fdaf-4960-8022-016578cfd31f" (UID: "e24a9dff-fdaf-4960-8022-016578cfd31f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:25:13.485791 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.485716    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:13.485791 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.485780    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e24a9dff-fdaf-4960-8022-016578cfd31f-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:13.485791 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:13.485794    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4szr\" (UniqueName: \"kubernetes.io/projected/e24a9dff-fdaf-4960-8022-016578cfd31f-kube-api-access-t4szr\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:14.096434 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:14.096405    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6"
Jul 09 16:25:14.096593 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:14.096400    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nr9g6" event={"ID":"e24a9dff-fdaf-4960-8022-016578cfd31f","Type":"ContainerDied","Data":"29da0d95a76bef3be06fcde365b9d608c98390868156279368325b06ba53e969"}
Jul 09 16:25:14.096593 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:14.096539    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29da0d95a76bef3be06fcde365b9d608c98390868156279368325b06ba53e969"
Jul 09 16:25:20.142844 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.142812    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"]
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143078    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="extract"
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143088    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="extract"
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143103    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="util"
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143108    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="util"
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143125    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="pull"
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143131    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="pull"
Jul 09 16:25:20.143219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.143176    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e24a9dff-fdaf-4960-8022-016578cfd31f" containerName="extract"
Jul 09 16:25:20.175598 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.175563    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"]
Jul 09 16:25:20.175730 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.175654    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.178238 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.178174    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 09 16:25:20.178427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.178410    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-z98f5\""
Jul 09 16:25:20.178477 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.178437    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 09 16:25:20.229644 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.229602    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5b9221b-7c21-4d3b-8538-71187ae4b137-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z4z9m\" (UID: \"f5b9221b-7c21-4d3b-8538-71187ae4b137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.229863 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.229690    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw64p\" (UniqueName: \"kubernetes.io/projected/f5b9221b-7c21-4d3b-8538-71187ae4b137-kube-api-access-cw64p\") pod \"cert-manager-operator-controller-manager-55c444bf86-z4z9m\" (UID: \"f5b9221b-7c21-4d3b-8538-71187ae4b137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.330400 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.330358    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw64p\" (UniqueName: \"kubernetes.io/projected/f5b9221b-7c21-4d3b-8538-71187ae4b137-kube-api-access-cw64p\") pod \"cert-manager-operator-controller-manager-55c444bf86-z4z9m\" (UID: \"f5b9221b-7c21-4d3b-8538-71187ae4b137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.330527 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.330438    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5b9221b-7c21-4d3b-8538-71187ae4b137-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z4z9m\" (UID: \"f5b9221b-7c21-4d3b-8538-71187ae4b137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.330845 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.330827    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5b9221b-7c21-4d3b-8538-71187ae4b137-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z4z9m\" (UID: \"f5b9221b-7c21-4d3b-8538-71187ae4b137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.340017 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.339986    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw64p\" (UniqueName: \"kubernetes.io/projected/f5b9221b-7c21-4d3b-8538-71187ae4b137-kube-api-access-cw64p\") pod \"cert-manager-operator-controller-manager-55c444bf86-z4z9m\" (UID: \"f5b9221b-7c21-4d3b-8538-71187ae4b137\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.484635 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.484551    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"
Jul 09 16:25:20.644367 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:20.644329    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m"]
Jul 09 16:25:20.649148 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:25:20.649119    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b9221b_7c21_4d3b_8538_71187ae4b137.slice/crio-77b71e02deedc4fc55277dc3a520bc0a3cc71d49ee85b7c8dfc00d4ef4fdbdfc WatchSource:0}: Error finding container 77b71e02deedc4fc55277dc3a520bc0a3cc71d49ee85b7c8dfc00d4ef4fdbdfc: Status 404 returned error can't find the container with id 77b71e02deedc4fc55277dc3a520bc0a3cc71d49ee85b7c8dfc00d4ef4fdbdfc
Jul 09 16:25:21.122139 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:21.122100    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m" event={"ID":"f5b9221b-7c21-4d3b-8538-71187ae4b137","Type":"ContainerStarted","Data":"77b71e02deedc4fc55277dc3a520bc0a3cc71d49ee85b7c8dfc00d4ef4fdbdfc"}
Jul 09 16:25:23.133443 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.133407    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m" event={"ID":"f5b9221b-7c21-4d3b-8538-71187ae4b137","Type":"ContainerStarted","Data":"91b0c87b45779cf1dfa85eca3f42d8527cef7517a1ab021804d87ba169802ae9"}
Jul 09 16:25:23.158366 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.158314    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z4z9m" podStartSLOduration=1.080218202 podStartE2EDuration="3.158295044s" totalImagesPullingTime="2.078076842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:25:23.156071002 +0000 UTC m=+334.082025283" watchObservedRunningTime="2026-07-09 16:25:23.158295044 +0000 UTC m=+334.084249321"
Jul 09 16:25:23.684111 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.684072    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f5w8j"]
Jul 09 16:25:23.708389 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.708339    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f5w8j"]
Jul 09 16:25:23.708578 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.708484    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:23.711172 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.711145    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 16:25:23.711301 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.711284    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-p5967\""
Jul 09 16:25:23.712033 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.712017    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 16:25:23.752491 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.752454    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67c2\" (UniqueName: \"kubernetes.io/projected/e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e-kube-api-access-k67c2\") pod \"cert-manager-webhook-6486ffd997-f5w8j\" (UID: \"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:23.752491 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.752490    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f5w8j\" (UID: \"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:23.853273 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.853229    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67c2\" (UniqueName: \"kubernetes.io/projected/e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e-kube-api-access-k67c2\") pod \"cert-manager-webhook-6486ffd997-f5w8j\" (UID: \"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:23.853273 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.853276    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f5w8j\" (UID: \"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:23.867166 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.867134    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67c2\" (UniqueName: \"kubernetes.io/projected/e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e-kube-api-access-k67c2\") pod \"cert-manager-webhook-6486ffd997-f5w8j\" (UID: \"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:23.872828 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:23.872795    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f5w8j\" (UID: \"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:24.030875 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.030785    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:24.184415 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.184383    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f5w8j"]
Jul 09 16:25:24.187509 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:25:24.187479    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2b4d3b7_3ba9_43bd_b711_23049ccb6a7e.slice/crio-c6998d87df063386aa1ff11ee86f2671c48dec3dc793391248a1c0605020e8c2 WatchSource:0}: Error finding container c6998d87df063386aa1ff11ee86f2671c48dec3dc793391248a1c0605020e8c2: Status 404 returned error can't find the container with id c6998d87df063386aa1ff11ee86f2671c48dec3dc793391248a1c0605020e8c2
Jul 09 16:25:24.431800 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.431746    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"]
Jul 09 16:25:24.434576 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.434557    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.437605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.437583    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n5v5x\""
Jul 09 16:25:24.437955 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.437942    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 16:25:24.438404 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.438392    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 16:25:24.446042 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.446018    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"]
Jul 09 16:25:24.458356 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.458331    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.458487 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.458360    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9mmt\" (UniqueName: \"kubernetes.io/projected/8de20afa-468e-423c-9c3d-9a3cdd9b2753-kube-api-access-m9mmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.458487 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.458428    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.559660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.559628    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.559862 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.559664    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9mmt\" (UniqueName: \"kubernetes.io/projected/8de20afa-468e-423c-9c3d-9a3cdd9b2753-kube-api-access-m9mmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.559862 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.559697    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.560137 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.560115    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.560181 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.560115    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.573779 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.573726    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9mmt\" (UniqueName: \"kubernetes.io/projected/8de20afa-468e-423c-9c3d-9a3cdd9b2753-kube-api-access-m9mmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.744024 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.743938    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:24.871145 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:24.871120    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"]
Jul 09 16:25:24.873820 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:25:24.873792    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de20afa_468e_423c_9c3d_9a3cdd9b2753.slice/crio-73baa1bc4837792f32e5ea0b3b8f2a37101aabc05f7fa081ccffe599a997355b WatchSource:0}: Error finding container 73baa1bc4837792f32e5ea0b3b8f2a37101aabc05f7fa081ccffe599a997355b: Status 404 returned error can't find the container with id 73baa1bc4837792f32e5ea0b3b8f2a37101aabc05f7fa081ccffe599a997355b
Jul 09 16:25:25.145252 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:25.145214    2586 generic.go:356] "Generic (PLEG): container finished" podID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerID="91bf3915769c7385054c644e6cae91b4e52c4820b5bb784482425a3a1054771a" exitCode=0
Jul 09 16:25:25.145442 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:25.145298    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq" event={"ID":"8de20afa-468e-423c-9c3d-9a3cdd9b2753","Type":"ContainerDied","Data":"91bf3915769c7385054c644e6cae91b4e52c4820b5bb784482425a3a1054771a"}
Jul 09 16:25:25.145442 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:25.145366    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq" event={"ID":"8de20afa-468e-423c-9c3d-9a3cdd9b2753","Type":"ContainerStarted","Data":"73baa1bc4837792f32e5ea0b3b8f2a37101aabc05f7fa081ccffe599a997355b"}
Jul 09 16:25:25.146981 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:25.146775    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j" event={"ID":"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e","Type":"ContainerStarted","Data":"c6998d87df063386aa1ff11ee86f2671c48dec3dc793391248a1c0605020e8c2"}
Jul 09 16:25:27.219937 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.219904    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k227t"]
Jul 09 16:25:27.225936 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.225912    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.228548 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.228524    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5bzzd\""
Jul 09 16:25:27.236451 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.236315    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k227t"]
Jul 09 16:25:27.286127 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.286089    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jnnh\" (UniqueName: \"kubernetes.io/projected/76facc19-70a3-47b0-a12a-96830924c421-kube-api-access-5jnnh\") pod \"cert-manager-cainjector-559cdd8c87-k227t\" (UID: \"76facc19-70a3-47b0-a12a-96830924c421\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.286127 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.286133    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76facc19-70a3-47b0-a12a-96830924c421-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k227t\" (UID: \"76facc19-70a3-47b0-a12a-96830924c421\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.387184 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.387146    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jnnh\" (UniqueName: \"kubernetes.io/projected/76facc19-70a3-47b0-a12a-96830924c421-kube-api-access-5jnnh\") pod \"cert-manager-cainjector-559cdd8c87-k227t\" (UID: \"76facc19-70a3-47b0-a12a-96830924c421\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.387375 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.387206    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76facc19-70a3-47b0-a12a-96830924c421-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k227t\" (UID: \"76facc19-70a3-47b0-a12a-96830924c421\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.400853 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.400824    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76facc19-70a3-47b0-a12a-96830924c421-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-k227t\" (UID: \"76facc19-70a3-47b0-a12a-96830924c421\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.401096 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.401074    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jnnh\" (UniqueName: \"kubernetes.io/projected/76facc19-70a3-47b0-a12a-96830924c421-kube-api-access-5jnnh\") pod \"cert-manager-cainjector-559cdd8c87-k227t\" (UID: \"76facc19-70a3-47b0-a12a-96830924c421\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.538280 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.538193    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t"
Jul 09 16:25:27.838402 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:27.838363    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-k227t"]
Jul 09 16:25:27.875613 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:25:27.875586    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76facc19_70a3_47b0_a12a_96830924c421.slice/crio-65b03797569b926fba59a35e7148e87ab770289f284291fbc16c685f67114f9f WatchSource:0}: Error finding container 65b03797569b926fba59a35e7148e87ab770289f284291fbc16c685f67114f9f: Status 404 returned error can't find the container with id 65b03797569b926fba59a35e7148e87ab770289f284291fbc16c685f67114f9f
Jul 09 16:25:28.164465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.164426    2586 generic.go:356] "Generic (PLEG): container finished" podID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerID="cbccc02df1183d4dc26cf75cce7895258e14b133adae18414d12018ff0bd4384" exitCode=0
Jul 09 16:25:28.164678 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.164515    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq" event={"ID":"8de20afa-468e-423c-9c3d-9a3cdd9b2753","Type":"ContainerDied","Data":"cbccc02df1183d4dc26cf75cce7895258e14b133adae18414d12018ff0bd4384"}
Jul 09 16:25:28.166274 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.166249    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t" event={"ID":"76facc19-70a3-47b0-a12a-96830924c421","Type":"ContainerStarted","Data":"7af997ccaa76ebe0de70ebb1714996c47552c02a44164ad14cbf25f59b20a7d7"}
Jul 09 16:25:28.166375 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.166288    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t" event={"ID":"76facc19-70a3-47b0-a12a-96830924c421","Type":"ContainerStarted","Data":"65b03797569b926fba59a35e7148e87ab770289f284291fbc16c685f67114f9f"}
Jul 09 16:25:28.168189 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.168165    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j" event={"ID":"e2b4d3b7-3ba9-43bd-b711-23049ccb6a7e","Type":"ContainerStarted","Data":"7bc33666c4cfc1c95e1a24768cec6ff8388c60e38b3d380f16d962b438b97eb7"}
Jul 09 16:25:28.168312 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.168295    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:28.207777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.207597    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j" podStartSLOduration=1.698458664 podStartE2EDuration="5.20757233s" totalImagesPullingTime="3.509113666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:25:28.205954057 +0000 UTC m=+339.131908336" watchObservedRunningTime="2026-07-09 16:25:28.20757233 +0000 UTC m=+339.133526609"
Jul 09 16:25:28.227034 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:28.226987    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-k227t" podStartSLOduration=1.22696799 podStartE2EDuration="1.22696799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:25:28.224999437 +0000 UTC m=+339.150953719" watchObservedRunningTime="2026-07-09 16:25:28.22696799 +0000 UTC m=+339.152922270"
Jul 09 16:25:29.175120 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:29.175082    2586 generic.go:356] "Generic (PLEG): container finished" podID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerID="666bcb6fb1a0dad54e5c5f68a36bf16257ad883123fa244e5f073bb0a848f26c" exitCode=0
Jul 09 16:25:29.175290 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:29.175168    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq" event={"ID":"8de20afa-468e-423c-9c3d-9a3cdd9b2753","Type":"ContainerDied","Data":"666bcb6fb1a0dad54e5c5f68a36bf16257ad883123fa244e5f073bb0a848f26c"}
Jul 09 16:25:30.306169 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.306145    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:30.412885 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.412848    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-util\") pod \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") "
Jul 09 16:25:30.412885 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.412898    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-bundle\") pod \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") "
Jul 09 16:25:30.413129 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.412924    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9mmt\" (UniqueName: \"kubernetes.io/projected/8de20afa-468e-423c-9c3d-9a3cdd9b2753-kube-api-access-m9mmt\") pod \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\" (UID: \"8de20afa-468e-423c-9c3d-9a3cdd9b2753\") "
Jul 09 16:25:30.413290 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.413253    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-bundle" (OuterVolumeSpecName: "bundle") pod "8de20afa-468e-423c-9c3d-9a3cdd9b2753" (UID: "8de20afa-468e-423c-9c3d-9a3cdd9b2753"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:25:30.415020 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.414989    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de20afa-468e-423c-9c3d-9a3cdd9b2753-kube-api-access-m9mmt" (OuterVolumeSpecName: "kube-api-access-m9mmt") pod "8de20afa-468e-423c-9c3d-9a3cdd9b2753" (UID: "8de20afa-468e-423c-9c3d-9a3cdd9b2753"). InnerVolumeSpecName "kube-api-access-m9mmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:25:30.514051 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.513974    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:30.514051 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.514004    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9mmt\" (UniqueName: \"kubernetes.io/projected/8de20afa-468e-423c-9c3d-9a3cdd9b2753-kube-api-access-m9mmt\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:30.674249 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.674206    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-util" (OuterVolumeSpecName: "util") pod "8de20afa-468e-423c-9c3d-9a3cdd9b2753" (UID: "8de20afa-468e-423c-9c3d-9a3cdd9b2753"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:25:30.715271 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:30.715237    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8de20afa-468e-423c-9c3d-9a3cdd9b2753-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:31.185043 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:31.185011    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq" event={"ID":"8de20afa-468e-423c-9c3d-9a3cdd9b2753","Type":"ContainerDied","Data":"73baa1bc4837792f32e5ea0b3b8f2a37101aabc05f7fa081ccffe599a997355b"}
Jul 09 16:25:31.185209 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:31.185059    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73baa1bc4837792f32e5ea0b3b8f2a37101aabc05f7fa081ccffe599a997355b"
Jul 09 16:25:31.185209 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:31.185023    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6x2qq"
Jul 09 16:25:34.182106 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:34.182066    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f5w8j"
Jul 09 16:25:36.833773 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.833726    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"]
Jul 09 16:25:36.834163 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834074    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="util"
Jul 09 16:25:36.834163 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834087    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="util"
Jul 09 16:25:36.834163 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834102    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="extract"
Jul 09 16:25:36.834163 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834110    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="extract"
Jul 09 16:25:36.834163 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834131    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="pull"
Jul 09 16:25:36.834163 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834137    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="pull"
Jul 09 16:25:36.834336 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.834189    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de20afa-468e-423c-9c3d-9a3cdd9b2753" containerName="extract"
Jul 09 16:25:36.836972 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.836956    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:36.839560 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.839535    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 16:25:36.839665 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.839536    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 16:25:36.840386 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.840370    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wqhpj\""
Jul 09 16:25:36.845989 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.845967    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"]
Jul 09 16:25:36.858548 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.858516    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsdwf\" (UniqueName: \"kubernetes.io/projected/8fa40253-60f5-43aa-a724-ee6da7be1f43-kube-api-access-hsdwf\") pod \"openshift-lws-operator-fd8ccff4c-742mk\" (UID: \"8fa40253-60f5-43aa-a724-ee6da7be1f43\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:36.858675 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.858607    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa40253-60f5-43aa-a724-ee6da7be1f43-tmp\") pod \"openshift-lws-operator-fd8ccff4c-742mk\" (UID: \"8fa40253-60f5-43aa-a724-ee6da7be1f43\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:36.959936 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.959893    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa40253-60f5-43aa-a724-ee6da7be1f43-tmp\") pod \"openshift-lws-operator-fd8ccff4c-742mk\" (UID: \"8fa40253-60f5-43aa-a724-ee6da7be1f43\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:36.960097 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.959970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsdwf\" (UniqueName: \"kubernetes.io/projected/8fa40253-60f5-43aa-a724-ee6da7be1f43-kube-api-access-hsdwf\") pod \"openshift-lws-operator-fd8ccff4c-742mk\" (UID: \"8fa40253-60f5-43aa-a724-ee6da7be1f43\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:36.960282 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.960262    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa40253-60f5-43aa-a724-ee6da7be1f43-tmp\") pod \"openshift-lws-operator-fd8ccff4c-742mk\" (UID: \"8fa40253-60f5-43aa-a724-ee6da7be1f43\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:36.968752 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:36.968719    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsdwf\" (UniqueName: \"kubernetes.io/projected/8fa40253-60f5-43aa-a724-ee6da7be1f43-kube-api-access-hsdwf\") pod \"openshift-lws-operator-fd8ccff4c-742mk\" (UID: \"8fa40253-60f5-43aa-a724-ee6da7be1f43\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:37.147185 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:37.147150    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"
Jul 09 16:25:37.276734 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:37.276709    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk"]
Jul 09 16:25:37.279659 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:25:37.279632    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fa40253_60f5_43aa_a724_ee6da7be1f43.slice/crio-a6fdb74cd7606a3db176b6429d76b68cb969055176ada9484ff9f67ba042f96e WatchSource:0}: Error finding container a6fdb74cd7606a3db176b6429d76b68cb969055176ada9484ff9f67ba042f96e: Status 404 returned error can't find the container with id a6fdb74cd7606a3db176b6429d76b68cb969055176ada9484ff9f67ba042f96e
Jul 09 16:25:38.223707 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:38.223657    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk" event={"ID":"8fa40253-60f5-43aa-a724-ee6da7be1f43","Type":"ContainerStarted","Data":"a6fdb74cd7606a3db176b6429d76b68cb969055176ada9484ff9f67ba042f96e"}
Jul 09 16:25:39.229239 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:39.229201    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk" event={"ID":"8fa40253-60f5-43aa-a724-ee6da7be1f43","Type":"ContainerStarted","Data":"78a8cbd0b79b721113cc3c53010ebe7d75cded26c2dbb1d18188fce8d4cab899"}
Jul 09 16:25:39.248212 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:39.248164    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-742mk" podStartSLOduration=1.7588421090000002 podStartE2EDuration="3.248147675s" totalImagesPullingTime="1.489305566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:25:39.245924442 +0000 UTC m=+350.171878730" watchObservedRunningTime="2026-07-09 16:25:39.248147675 +0000 UTC m=+350.174101952"
Jul 09 16:25:42.324351 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.324269    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"]
Jul 09 16:25:42.326542 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.326525    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.328991 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.328970    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-n5v5x\""
Jul 09 16:25:42.328991 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.328978    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 16:25:42.329133 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.328976    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 16:25:42.336958 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.336937    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"]
Jul 09 16:25:42.403899 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.403858    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.404081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.403969    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmcq2\" (UniqueName: \"kubernetes.io/projected/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-kube-api-access-qmcq2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.404081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.403995    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.505203 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.505148    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmcq2\" (UniqueName: \"kubernetes.io/projected/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-kube-api-access-qmcq2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.505203 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.505206    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.505421 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.505236    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.505687 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.505666    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.505687 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.505680    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.518942 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.518901    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmcq2\" (UniqueName: \"kubernetes.io/projected/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-kube-api-access-qmcq2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.635935 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.635895    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:42.773067 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:42.773045    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"]
Jul 09 16:25:42.775027 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:25:42.774998    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fa92a8_bd2e_4593_92eb_1a7d0be93f1c.slice/crio-166a1d95cff39ea384c556e8699b26d0dcc14fc1bac80a420aa9c97c3cdd679d WatchSource:0}: Error finding container 166a1d95cff39ea384c556e8699b26d0dcc14fc1bac80a420aa9c97c3cdd679d: Status 404 returned error can't find the container with id 166a1d95cff39ea384c556e8699b26d0dcc14fc1bac80a420aa9c97c3cdd679d
Jul 09 16:25:43.247826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:43.247794    2586 generic.go:356] "Generic (PLEG): container finished" podID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerID="e2fd2a2a5d1200f37f129a8b71f362366a40564deaffb288db5431956bf43ce7" exitCode=0
Jul 09 16:25:43.247967 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:43.247875    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m" event={"ID":"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c","Type":"ContainerDied","Data":"e2fd2a2a5d1200f37f129a8b71f362366a40564deaffb288db5431956bf43ce7"}
Jul 09 16:25:43.247967 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:43.247927    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m" event={"ID":"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c","Type":"ContainerStarted","Data":"166a1d95cff39ea384c556e8699b26d0dcc14fc1bac80a420aa9c97c3cdd679d"}
Jul 09 16:25:45.258531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:45.258500    2586 generic.go:356] "Generic (PLEG): container finished" podID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerID="8eb700d74c05d5b5c25b657e4ff17590e542bcf11abe9c07e0099e01191cf861" exitCode=0
Jul 09 16:25:45.258969 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:45.258584    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m" event={"ID":"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c","Type":"ContainerDied","Data":"8eb700d74c05d5b5c25b657e4ff17590e542bcf11abe9c07e0099e01191cf861"}
Jul 09 16:25:46.266205 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:46.266167    2586 generic.go:356] "Generic (PLEG): container finished" podID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerID="aa7f47dfced447cf7b7284ba0fa483ffd4ab345cef40bab2e39c96c5ddab6954" exitCode=0
Jul 09 16:25:46.266620 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:46.266211    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m" event={"ID":"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c","Type":"ContainerDied","Data":"aa7f47dfced447cf7b7284ba0fa483ffd4ab345cef40bab2e39c96c5ddab6954"}
Jul 09 16:25:47.396157 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.396132    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:47.445799 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.445745    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-bundle\") pod \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") "
Jul 09 16:25:47.445989 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.445819    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-util\") pod \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") "
Jul 09 16:25:47.445989 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.445882    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmcq2\" (UniqueName: \"kubernetes.io/projected/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-kube-api-access-qmcq2\") pod \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\" (UID: \"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c\") "
Jul 09 16:25:47.446498 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.446466    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-bundle" (OuterVolumeSpecName: "bundle") pod "20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" (UID: "20fa92a8-bd2e-4593-92eb-1a7d0be93f1c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:25:47.448034 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.447998    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-kube-api-access-qmcq2" (OuterVolumeSpecName: "kube-api-access-qmcq2") pod "20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" (UID: "20fa92a8-bd2e-4593-92eb-1a7d0be93f1c"). InnerVolumeSpecName "kube-api-access-qmcq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:25:47.451914 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.451875    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-util" (OuterVolumeSpecName: "util") pod "20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" (UID: "20fa92a8-bd2e-4593-92eb-1a7d0be93f1c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:25:47.547187 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.547103    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:47.547187 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.547131    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:47.547187 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:47.547140    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmcq2\" (UniqueName: \"kubernetes.io/projected/20fa92a8-bd2e-4593-92eb-1a7d0be93f1c-kube-api-access-qmcq2\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:25:48.277295 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:48.277256    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m" event={"ID":"20fa92a8-bd2e-4593-92eb-1a7d0be93f1c","Type":"ContainerDied","Data":"166a1d95cff39ea384c556e8699b26d0dcc14fc1bac80a420aa9c97c3cdd679d"}
Jul 09 16:25:48.277295 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:48.277282    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdwf9m"
Jul 09 16:25:48.277295 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:25:48.277303    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="166a1d95cff39ea384c556e8699b26d0dcc14fc1bac80a420aa9c97c3cdd679d"
Jul 09 16:26:00.179117 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179066    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"]
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179435    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="pull"
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179447    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="pull"
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179457    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="extract"
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179462    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="extract"
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179471    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="util"
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179476    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="util"
Jul 09 16:26:00.179566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.179532    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fa92a8-bd2e-4593-92eb-1a7d0be93f1c" containerName="extract"
Jul 09 16:26:00.182361 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.182344    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.185271 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.185248    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 16:26:00.185392 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.185270    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kb6qk\""
Jul 09 16:26:00.185623 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.185610    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 16:26:00.185818 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.185799    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 16:26:00.185883 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.185870    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 16:26:00.198556 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.198530    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"]
Jul 09 16:26:00.254978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.254938    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb6af1b2-ed62-4334-a45f-2074c48228d2-tmp\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.254978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.254979    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpmjw\" (UniqueName: \"kubernetes.io/projected/fb6af1b2-ed62-4334-a45f-2074c48228d2-kube-api-access-xpmjw\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.255228 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.255008    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb6af1b2-ed62-4334-a45f-2074c48228d2-manifests\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.255228 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.255080    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb6af1b2-ed62-4334-a45f-2074c48228d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.255228 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.255099    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb6af1b2-ed62-4334-a45f-2074c48228d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.355673 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.355636    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb6af1b2-ed62-4334-a45f-2074c48228d2-tmp\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.355673 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.355676    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpmjw\" (UniqueName: \"kubernetes.io/projected/fb6af1b2-ed62-4334-a45f-2074c48228d2-kube-api-access-xpmjw\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.355904 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.355798    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb6af1b2-ed62-4334-a45f-2074c48228d2-manifests\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.355904 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.355858    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb6af1b2-ed62-4334-a45f-2074c48228d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.355904 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.355888    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb6af1b2-ed62-4334-a45f-2074c48228d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.356160 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.356138    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fb6af1b2-ed62-4334-a45f-2074c48228d2-manifests\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.357940 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.357917    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb6af1b2-ed62-4334-a45f-2074c48228d2-tmp\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.358258 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.358239    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb6af1b2-ed62-4334-a45f-2074c48228d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.358386 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.358363    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fb6af1b2-ed62-4334-a45f-2074c48228d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.365090 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.365066    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpmjw\" (UniqueName: \"kubernetes.io/projected/fb6af1b2-ed62-4334-a45f-2074c48228d2-kube-api-access-xpmjw\") pod \"opendatahub-operator-controller-manager-5bd696597f-k5htn\" (UID: \"fb6af1b2-ed62-4334-a45f-2074c48228d2\") " pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.493048 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.492950    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:00.631895 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:00.631869    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"]
Jul 09 16:26:00.634540 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:26:00.634512    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6af1b2_ed62_4334_a45f_2074c48228d2.slice/crio-b97c8d37dbaf6e89987bf8a7e089b083ca10cfb2fcb6b12fe5aab5f58482e4c8 WatchSource:0}: Error finding container b97c8d37dbaf6e89987bf8a7e089b083ca10cfb2fcb6b12fe5aab5f58482e4c8: Status 404 returned error can't find the container with id b97c8d37dbaf6e89987bf8a7e089b083ca10cfb2fcb6b12fe5aab5f58482e4c8
Jul 09 16:26:01.334087 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:01.334051    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn" event={"ID":"fb6af1b2-ed62-4334-a45f-2074c48228d2","Type":"ContainerStarted","Data":"b97c8d37dbaf6e89987bf8a7e089b083ca10cfb2fcb6b12fe5aab5f58482e4c8"}
Jul 09 16:26:04.355528 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:04.355493    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn" event={"ID":"fb6af1b2-ed62-4334-a45f-2074c48228d2","Type":"ContainerStarted","Data":"90abb875e5678be7dfcd053c0c64e421b73bc77bde090c94d83025f4996a73ad"}
Jul 09 16:26:05.361781 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:05.361731    2586 generic.go:356] "Generic (PLEG): container finished" podID="fb6af1b2-ed62-4334-a45f-2074c48228d2" containerID="90abb875e5678be7dfcd053c0c64e421b73bc77bde090c94d83025f4996a73ad" exitCode=0
Jul 09 16:26:05.362176 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:05.361807    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn" event={"ID":"fb6af1b2-ed62-4334-a45f-2074c48228d2","Type":"ContainerDied","Data":"90abb875e5678be7dfcd053c0c64e421b73bc77bde090c94d83025f4996a73ad"}
Jul 09 16:26:06.368291 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:06.368253    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn" event={"ID":"fb6af1b2-ed62-4334-a45f-2074c48228d2","Type":"ContainerStarted","Data":"a174f3d907a935a3980cf428f8d33ac115a6d888b178bb4d5ecf6f05834f6f73"}
Jul 09 16:26:06.368677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:06.368382    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:06.403533 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:06.403484    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn" podStartSLOduration=1.5117883989999998 podStartE2EDuration="6.403465377s" totalImagesPullingTime="3.780643821s" totalInitContainerRuntime="1.111033157s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:26:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:26:06.402360499 +0000 UTC m=+377.328314776" watchObservedRunningTime="2026-07-09 16:26:06.403465377 +0000 UTC m=+377.329419700"
Jul 09 16:26:17.374718 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:17.374681    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bd696597f-k5htn"
Jul 09 16:26:22.572253 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.572217    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kbwqh"]
Jul 09 16:26:22.574578 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.574560    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.577386 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.577365    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-855tx\""
Jul 09 16:26:22.577646 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.577631    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 09 16:26:22.596246 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.596223    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kbwqh"]
Jul 09 16:26:22.642747 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.642710    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa630290-6a31-4db5-a88d-d87afa2b19f4-cert\") pod \"odh-model-controller-7998c67dc4-kbwqh\" (UID: \"fa630290-6a31-4db5-a88d-d87afa2b19f4\") " pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.642924 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.642787    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtnbw\" (UniqueName: \"kubernetes.io/projected/fa630290-6a31-4db5-a88d-d87afa2b19f4-kube-api-access-gtnbw\") pod \"odh-model-controller-7998c67dc4-kbwqh\" (UID: \"fa630290-6a31-4db5-a88d-d87afa2b19f4\") " pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.744014 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.743973    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa630290-6a31-4db5-a88d-d87afa2b19f4-cert\") pod \"odh-model-controller-7998c67dc4-kbwqh\" (UID: \"fa630290-6a31-4db5-a88d-d87afa2b19f4\") " pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.744202 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.744029    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtnbw\" (UniqueName: \"kubernetes.io/projected/fa630290-6a31-4db5-a88d-d87afa2b19f4-kube-api-access-gtnbw\") pod \"odh-model-controller-7998c67dc4-kbwqh\" (UID: \"fa630290-6a31-4db5-a88d-d87afa2b19f4\") " pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.746391 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.746365    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa630290-6a31-4db5-a88d-d87afa2b19f4-cert\") pod \"odh-model-controller-7998c67dc4-kbwqh\" (UID: \"fa630290-6a31-4db5-a88d-d87afa2b19f4\") " pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.761524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.761495    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtnbw\" (UniqueName: \"kubernetes.io/projected/fa630290-6a31-4db5-a88d-d87afa2b19f4-kube-api-access-gtnbw\") pod \"odh-model-controller-7998c67dc4-kbwqh\" (UID: \"fa630290-6a31-4db5-a88d-d87afa2b19f4\") " pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:22.884660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:22.884622    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:23.037328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.037285    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kbwqh"]
Jul 09 16:26:23.043399 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:26:23.043368    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa630290_6a31_4db5_a88d_d87afa2b19f4.slice/crio-292884b8dc05a5898f11f0948d5f399805b04b0ae1e553e1569e2bd8bde9648d WatchSource:0}: Error finding container 292884b8dc05a5898f11f0948d5f399805b04b0ae1e553e1569e2bd8bde9648d: Status 404 returned error can't find the container with id 292884b8dc05a5898f11f0948d5f399805b04b0ae1e553e1569e2bd8bde9648d
Jul 09 16:26:23.443546 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.443508    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh" event={"ID":"fa630290-6a31-4db5-a88d-d87afa2b19f4","Type":"ContainerStarted","Data":"292884b8dc05a5898f11f0948d5f399805b04b0ae1e553e1569e2bd8bde9648d"}
Jul 09 16:26:23.589439 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.589401    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9nz6j"]
Jul 09 16:26:23.592596 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.592575    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:23.595105 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.595082    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 09 16:26:23.605412 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.605388    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9nz6j"]
Jul 09 16:26:23.650700 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.650660    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7895c73c-a4ad-41f2-be7b-969dfec89911-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:23.650914 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.650725    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr4s2\" (UniqueName: \"kubernetes.io/projected/7895c73c-a4ad-41f2-be7b-969dfec89911-kube-api-access-kr4s2\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:23.751925 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.751835    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr4s2\" (UniqueName: \"kubernetes.io/projected/7895c73c-a4ad-41f2-be7b-969dfec89911-kube-api-access-kr4s2\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:23.752071 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.751958    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7895c73c-a4ad-41f2-be7b-969dfec89911-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:23.752110 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:26:23.752092    2586 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 09 16:26:23.752177 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:26:23.752167    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7895c73c-a4ad-41f2-be7b-969dfec89911-tls-certs podName:7895c73c-a4ad-41f2-be7b-969dfec89911 nodeName:}" failed. No retries permitted until 2026-07-09 16:26:24.252150001 +0000 UTC m=+395.178104256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7895c73c-a4ad-41f2-be7b-969dfec89911-tls-certs") pod "model-serving-api-54d89bd5fc-9nz6j" (UID: "7895c73c-a4ad-41f2-be7b-969dfec89911") : secret "model-serving-api-tls" not found
Jul 09 16:26:23.768607 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:23.768575    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr4s2\" (UniqueName: \"kubernetes.io/projected/7895c73c-a4ad-41f2-be7b-969dfec89911-kube-api-access-kr4s2\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:24.255645 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:24.255596    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7895c73c-a4ad-41f2-be7b-969dfec89911-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:24.258695 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:24.258655    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7895c73c-a4ad-41f2-be7b-969dfec89911-tls-certs\") pod \"model-serving-api-54d89bd5fc-9nz6j\" (UID: \"7895c73c-a4ad-41f2-be7b-969dfec89911\") " pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:24.503584 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:24.503532    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:24.661543 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:26:24.661506    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7895c73c_a4ad_41f2_be7b_969dfec89911.slice/crio-7632cb5f991414fad9a2890dd1096ceeece13b524e6f9c80cb7c76628e887651 WatchSource:0}: Error finding container 7632cb5f991414fad9a2890dd1096ceeece13b524e6f9c80cb7c76628e887651: Status 404 returned error can't find the container with id 7632cb5f991414fad9a2890dd1096ceeece13b524e6f9c80cb7c76628e887651
Jul 09 16:26:24.672123 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:24.672096    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9nz6j"]
Jul 09 16:26:25.455628 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:25.455582    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j" event={"ID":"7895c73c-a4ad-41f2-be7b-969dfec89911","Type":"ContainerStarted","Data":"7632cb5f991414fad9a2890dd1096ceeece13b524e6f9c80cb7c76628e887651"}
Jul 09 16:26:26.463282 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:26.463243    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh" event={"ID":"fa630290-6a31-4db5-a88d-d87afa2b19f4","Type":"ContainerStarted","Data":"bf174738bfe7519c9718238ae1a1ff25c8f78231be5097b28a80c2533a1d630b"}
Jul 09 16:26:26.463695 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:26.463438    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:26.480978 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:26.480910    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh" podStartSLOduration=1.4670668949999999 podStartE2EDuration="4.480893501s" totalImagesPullingTime="3.013826606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:26:26.480010548 +0000 UTC m=+397.405964827" watchObservedRunningTime="2026-07-09 16:26:26.480893501 +0000 UTC m=+397.406847779"
Jul 09 16:26:27.469422 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:27.469379    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j" event={"ID":"7895c73c-a4ad-41f2-be7b-969dfec89911","Type":"ContainerStarted","Data":"5a9ee61352c5757be8ab8df44aff392738fe4a083ba2d778157f6ceaf1351db7"}
Jul 09 16:26:27.469918 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:27.469457    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:27.495040 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:27.494985    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j" podStartSLOduration=2.337363816 podStartE2EDuration="4.494967675s" totalImagesPullingTime="2.157603859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:26:27.488991532 +0000 UTC m=+398.414945810" watchObservedRunningTime="2026-07-09 16:26:27.494967675 +0000 UTC m=+398.420921952"
Jul 09 16:26:28.158200 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.158163    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"]
Jul 09 16:26:28.161568 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.161550    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.166438 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.166397    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 16:26:28.166585 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.166470    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9pdjd\""
Jul 09 16:26:28.171447 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.166648    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 16:26:28.171447 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.166667    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 16:26:28.171447 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.166737    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 16:26:28.183675 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.183644    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"]
Jul 09 16:26:28.186934 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.186901    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eaa6961b-be84-4761-b668-ddd030e2c902-tmp\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.187107 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.186948    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaa6961b-be84-4761-b668-ddd030e2c902-tls-certs\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.187107 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.187019    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f57n\" (UniqueName: \"kubernetes.io/projected/eaa6961b-be84-4761-b668-ddd030e2c902-kube-api-access-9f57n\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.288108 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.288074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eaa6961b-be84-4761-b668-ddd030e2c902-tmp\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.288108 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.288115    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaa6961b-be84-4761-b668-ddd030e2c902-tls-certs\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.288314 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.288167    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f57n\" (UniqueName: \"kubernetes.io/projected/eaa6961b-be84-4761-b668-ddd030e2c902-kube-api-access-9f57n\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.290378 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.290340    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eaa6961b-be84-4761-b668-ddd030e2c902-tmp\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.290505 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.290448    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eaa6961b-be84-4761-b668-ddd030e2c902-tls-certs\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.296809 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.296783    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f57n\" (UniqueName: \"kubernetes.io/projected/eaa6961b-be84-4761-b668-ddd030e2c902-kube-api-access-9f57n\") pod \"kube-auth-proxy-868f9f885b-tpg7f\" (UID: \"eaa6961b-be84-4761-b668-ddd030e2c902\") " pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.477252 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.477174    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"
Jul 09 16:26:28.607432 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:28.607404    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f"]
Jul 09 16:26:28.609123 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:26:28.609098    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa6961b_be84_4761_b668_ddd030e2c902.slice/crio-e20da22c5bff9ae6d35c6163d2d614e9b7405aa7657ebddc21ede1a68fa55233 WatchSource:0}: Error finding container e20da22c5bff9ae6d35c6163d2d614e9b7405aa7657ebddc21ede1a68fa55233: Status 404 returned error can't find the container with id e20da22c5bff9ae6d35c6163d2d614e9b7405aa7657ebddc21ede1a68fa55233
Jul 09 16:26:29.480919 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.480881    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f" event={"ID":"eaa6961b-be84-4761-b668-ddd030e2c902","Type":"ContainerStarted","Data":"e20da22c5bff9ae6d35c6163d2d614e9b7405aa7657ebddc21ede1a68fa55233"}
Jul 09 16:26:29.778750 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.778657    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"]
Jul 09 16:26:29.782518 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.782489    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.788537 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.788266    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 16:26:29.788537 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.788305    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 16:26:29.788537 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.788306    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 16:26:29.788537 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.788423    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4gfbp\""
Jul 09 16:26:29.800063 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.800039    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"]
Jul 09 16:26:29.898427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.898141    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-cert\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.898427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.898207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkgg9\" (UniqueName: \"kubernetes.io/projected/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-kube-api-access-gkgg9\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.898427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.898244    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-manager-config\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.898427 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.898290    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-metrics-cert\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.999202 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.999167    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-manager-config\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.999405 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.999216    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-metrics-cert\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.999405 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.999263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-cert\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.999405 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.999290    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkgg9\" (UniqueName: \"kubernetes.io/projected/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-kube-api-access-gkgg9\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:29.999875 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:29.999856    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-manager-config\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:30.001739 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:30.001717    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-cert\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:30.001862 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:30.001782    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-metrics-cert\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:30.018589 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:30.018552    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkgg9\" (UniqueName: \"kubernetes.io/projected/55e78e6f-921d-46e6-a1a0-a6dae97c2afc-kube-api-access-gkgg9\") pod \"lws-controller-manager-7d5799cc79-pnzfj\" (UID: \"55e78e6f-921d-46e6-a1a0-a6dae97c2afc\") " pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:30.095122 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:30.095096    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:30.232433 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:30.232389    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"]
Jul 09 16:26:30.237300 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:26:30.237267    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e78e6f_921d_46e6_a1a0_a6dae97c2afc.slice/crio-1b7f9f5abe8999ab3696e9d2b05505f04cffdcdbf00655e8326f535162995401 WatchSource:0}: Error finding container 1b7f9f5abe8999ab3696e9d2b05505f04cffdcdbf00655e8326f535162995401: Status 404 returned error can't find the container with id 1b7f9f5abe8999ab3696e9d2b05505f04cffdcdbf00655e8326f535162995401
Jul 09 16:26:30.486504 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:30.486411    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj" event={"ID":"55e78e6f-921d-46e6-a1a0-a6dae97c2afc","Type":"ContainerStarted","Data":"1b7f9f5abe8999ab3696e9d2b05505f04cffdcdbf00655e8326f535162995401"}
Jul 09 16:26:32.499707 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:32.499672    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f" event={"ID":"eaa6961b-be84-4761-b668-ddd030e2c902","Type":"ContainerStarted","Data":"81e68ba0e23c1d8114553bdc1152686c323496f8d815038de86859ddddb45917"}
Jul 09 16:26:32.538324 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:32.538274    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-868f9f885b-tpg7f" podStartSLOduration=1.336589824 podStartE2EDuration="4.538254851s" totalImagesPullingTime="3.201665027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:26:32.536860073 +0000 UTC m=+403.462814352" watchObservedRunningTime="2026-07-09 16:26:32.538254851 +0000 UTC m=+403.464209128"
Jul 09 16:26:37.472069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:37.472033    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kbwqh"
Jul 09 16:26:38.477943 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:38.477914    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9nz6j"
Jul 09 16:26:50.587714 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:50.587666    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj" event={"ID":"55e78e6f-921d-46e6-a1a0-a6dae97c2afc","Type":"ContainerStarted","Data":"078ab18a720512a9b17bee5362fbe0f7d98fede9ee4660a28d0863d7f1842290"}
Jul 09 16:26:50.588164 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:50.587876    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:26:50.607698 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:26:50.607644    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj" podStartSLOduration=1.344155315 podStartE2EDuration="21.607627696s" totalImagesPullingTime="20.263472381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:26:50.605450277 +0000 UTC m=+421.531404578" watchObservedRunningTime="2026-07-09 16:26:50.607627696 +0000 UTC m=+421.533581974"
Jul 09 16:27:01.595234 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:01.595191    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7d5799cc79-pnzfj"
Jul 09 16:27:26.728651 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:26.728567    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75d5454df9-tsm28"]
Jul 09 16:27:35.804688 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.804651    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"]
Jul 09 16:27:35.807460 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.807437    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.810376 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.810344    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 16:27:35.810531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.810427    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 16:27:35.810778 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.810739    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n7pqw\""
Jul 09 16:27:35.820226 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.820204    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"]
Jul 09 16:27:35.866281 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.866232    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.866498 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.866304    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkvnf\" (UniqueName: \"kubernetes.io/projected/4fd28db9-ca9f-47bc-b417-b822a2a653af-kube-api-access-mkvnf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.866498 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.866412    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.967097 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.967057    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.967097 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.967096    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkvnf\" (UniqueName: \"kubernetes.io/projected/4fd28db9-ca9f-47bc-b417-b822a2a653af-kube-api-access-mkvnf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.967308 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.967138    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.967561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.967539    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.967598 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.967545    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:35.976186 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:35.976160    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkvnf\" (UniqueName: \"kubernetes.io/projected/4fd28db9-ca9f-47bc-b417-b822a2a653af-kube-api-access-mkvnf\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:36.116329 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.116295    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:36.250135 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.250111    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"]
Jul 09 16:27:36.252198 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:27:36.252170    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd28db9_ca9f_47bc_b417_b822a2a653af.slice/crio-1421256a4ae073f8b8f2777d8935b97df945bfb840820c89010926e2abbec337 WatchSource:0}: Error finding container 1421256a4ae073f8b8f2777d8935b97df945bfb840820c89010926e2abbec337: Status 404 returned error can't find the container with id 1421256a4ae073f8b8f2777d8935b97df945bfb840820c89010926e2abbec337
Jul 09 16:27:36.618311 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.618271    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"]
Jul 09 16:27:36.620905 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.620881    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.630682 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.630649    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"]
Jul 09 16:27:36.674456 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.674420    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljr7l\" (UniqueName: \"kubernetes.io/projected/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-kube-api-access-ljr7l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.674633 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.674476    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.674633 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.674496    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.775786 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.775733    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljr7l\" (UniqueName: \"kubernetes.io/projected/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-kube-api-access-ljr7l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.775963 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.775834    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.775963 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.775864    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.776212 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.776193    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.776249 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.776227    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.784754 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.784726    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljr7l\" (UniqueName: \"kubernetes.io/projected/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-kube-api-access-ljr7l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:36.802151 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.802115    2586 generic.go:356] "Generic (PLEG): container finished" podID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerID="3bb8a2a0a71a974f3e2c15a2829c13c87088185efd00fdff8792b46351e36773" exitCode=0
Jul 09 16:27:36.802319 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.802198    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr" event={"ID":"4fd28db9-ca9f-47bc-b417-b822a2a653af","Type":"ContainerDied","Data":"3bb8a2a0a71a974f3e2c15a2829c13c87088185efd00fdff8792b46351e36773"}
Jul 09 16:27:36.802319 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.802251    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr" event={"ID":"4fd28db9-ca9f-47bc-b417-b822a2a653af","Type":"ContainerStarted","Data":"1421256a4ae073f8b8f2777d8935b97df945bfb840820c89010926e2abbec337"}
Jul 09 16:27:36.930165 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:36.930075    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:37.064616 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.064591    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"]
Jul 09 16:27:37.066286 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:27:37.066249    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad61d77a_3a6f_4545_b92d_6e86bf8176e0.slice/crio-1a843f5551f7de3a830215d722172c75fe8caa0536793880f0b68c112198018b WatchSource:0}: Error finding container 1a843f5551f7de3a830215d722172c75fe8caa0536793880f0b68c112198018b: Status 404 returned error can't find the container with id 1a843f5551f7de3a830215d722172c75fe8caa0536793880f0b68c112198018b
Jul 09 16:27:37.192688 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.192611    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"]
Jul 09 16:27:37.195106 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.195088    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.206800 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.206747    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"]
Jul 09 16:27:37.280997 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.280963    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.280997 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.280997    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.281201 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.281025    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwjz\" (UniqueName: \"kubernetes.io/projected/1915b720-a446-4114-b498-d96f1292a493-kube-api-access-8mwjz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.381933 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.381894    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.381933 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.381932    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.382128 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.381957    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwjz\" (UniqueName: \"kubernetes.io/projected/1915b720-a446-4114-b498-d96f1292a493-kube-api-access-8mwjz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.382269 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.382248    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.382328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.382312    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.392267 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.392243    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwjz\" (UniqueName: \"kubernetes.io/projected/1915b720-a446-4114-b498-d96f1292a493-kube-api-access-8mwjz\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.531688 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.531598    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:37.744841 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.744811    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"]
Jul 09 16:27:37.755821 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:27:37.755787    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1915b720_a446_4114_b498_d96f1292a493.slice/crio-49e2c608cc28adbbf57dbe249811513a4a225d599bb42677f6ec819ce21cb61f WatchSource:0}: Error finding container 49e2c608cc28adbbf57dbe249811513a4a225d599bb42677f6ec819ce21cb61f: Status 404 returned error can't find the container with id 49e2c608cc28adbbf57dbe249811513a4a225d599bb42677f6ec819ce21cb61f
Jul 09 16:27:37.796753 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.796725    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"]
Jul 09 16:27:37.799857 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.799835    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.811424 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.811397    2586 generic.go:356] "Generic (PLEG): container finished" podID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerID="5023023b41c77993fa4a3779caed14f0dc03df437c8a6bec42c844e51447bf10" exitCode=0
Jul 09 16:27:37.811537 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.811478    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr" event={"ID":"4fd28db9-ca9f-47bc-b417-b822a2a653af","Type":"ContainerDied","Data":"5023023b41c77993fa4a3779caed14f0dc03df437c8a6bec42c844e51447bf10"}
Jul 09 16:27:37.812216 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.812193    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"]
Jul 09 16:27:37.813580 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.813556    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz" event={"ID":"1915b720-a446-4114-b498-d96f1292a493","Type":"ContainerStarted","Data":"49e2c608cc28adbbf57dbe249811513a4a225d599bb42677f6ec819ce21cb61f"}
Jul 09 16:27:37.815341 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.815321    2586 generic.go:356] "Generic (PLEG): container finished" podID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerID="55bc8a1860a3ae59017a06d3f59a1ee8de0e38e10f69895ab24248efb825d159" exitCode=0
Jul 09 16:27:37.815422 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.815387    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc" event={"ID":"ad61d77a-3a6f-4545-b92d-6e86bf8176e0","Type":"ContainerDied","Data":"55bc8a1860a3ae59017a06d3f59a1ee8de0e38e10f69895ab24248efb825d159"}
Jul 09 16:27:37.815422 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.815419    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc" event={"ID":"ad61d77a-3a6f-4545-b92d-6e86bf8176e0","Type":"ContainerStarted","Data":"1a843f5551f7de3a830215d722172c75fe8caa0536793880f0b68c112198018b"}
Jul 09 16:27:37.887119 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.887091    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.887258 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.887138    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc9tl\" (UniqueName: \"kubernetes.io/projected/8e65f840-c5f3-4ee5-9901-db6192eb8577-kube-api-access-mc9tl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.887324 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.887286    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.988278 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.988240    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.988655 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.988298    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.988655 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.988341    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9tl\" (UniqueName: \"kubernetes.io/projected/8e65f840-c5f3-4ee5-9901-db6192eb8577-kube-api-access-mc9tl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.988655 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.988637    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.988791 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.988662    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:37.997644 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:37.997623    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc9tl\" (UniqueName: \"kubernetes.io/projected/8e65f840-c5f3-4ee5-9901-db6192eb8577-kube-api-access-mc9tl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:38.109606 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.109572    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:38.260178 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:27:38.260145    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e65f840_c5f3_4ee5_9901_db6192eb8577.slice/crio-97a3d7e33c57b30af8297c154ee8fdce32e115dd8c000edb2663611a156d7c01 WatchSource:0}: Error finding container 97a3d7e33c57b30af8297c154ee8fdce32e115dd8c000edb2663611a156d7c01: Status 404 returned error can't find the container with id 97a3d7e33c57b30af8297c154ee8fdce32e115dd8c000edb2663611a156d7c01
Jul 09 16:27:38.260531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.260509    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"]
Jul 09 16:27:38.822241 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.822147    2586 generic.go:356] "Generic (PLEG): container finished" podID="1915b720-a446-4114-b498-d96f1292a493" containerID="af96b35c1ccf44cbe05c9f24f774284dcb41b47f3118a48887b5f77e94a67f37" exitCode=0
Jul 09 16:27:38.822241 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.822187    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz" event={"ID":"1915b720-a446-4114-b498-d96f1292a493","Type":"ContainerDied","Data":"af96b35c1ccf44cbe05c9f24f774284dcb41b47f3118a48887b5f77e94a67f37"}
Jul 09 16:27:38.824117 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.824095    2586 generic.go:356] "Generic (PLEG): container finished" podID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerID="3f32c762b4563ceb759c7900ee660c33ef97271438991dfb100909642bbac41e" exitCode=0
Jul 09 16:27:38.824230 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.824160    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc" event={"ID":"ad61d77a-3a6f-4545-b92d-6e86bf8176e0","Type":"ContainerDied","Data":"3f32c762b4563ceb759c7900ee660c33ef97271438991dfb100909642bbac41e"}
Jul 09 16:27:38.825750 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.825685    2586 generic.go:356] "Generic (PLEG): container finished" podID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerID="1f3c8e094c823e1b120a3f94a7499d4bbdbe435437d75a0c0339f06b8aa9d71c" exitCode=0
Jul 09 16:27:38.825835 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.825793    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6" event={"ID":"8e65f840-c5f3-4ee5-9901-db6192eb8577","Type":"ContainerDied","Data":"1f3c8e094c823e1b120a3f94a7499d4bbdbe435437d75a0c0339f06b8aa9d71c"}
Jul 09 16:27:38.825835 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.825829    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6" event={"ID":"8e65f840-c5f3-4ee5-9901-db6192eb8577","Type":"ContainerStarted","Data":"97a3d7e33c57b30af8297c154ee8fdce32e115dd8c000edb2663611a156d7c01"}
Jul 09 16:27:38.828203 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.828097    2586 generic.go:356] "Generic (PLEG): container finished" podID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerID="f003d866f80e050de8db8cdfffb89f8cb42cb19e0cfcb75b20f9ee4f72ff81a9" exitCode=0
Jul 09 16:27:38.828203 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:38.828165    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr" event={"ID":"4fd28db9-ca9f-47bc-b417-b822a2a653af","Type":"ContainerDied","Data":"f003d866f80e050de8db8cdfffb89f8cb42cb19e0cfcb75b20f9ee4f72ff81a9"}
Jul 09 16:27:39.835061 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.835032    2586 generic.go:356] "Generic (PLEG): container finished" podID="1915b720-a446-4114-b498-d96f1292a493" containerID="315ce255581291f909e5e901eef5ce4b46902aaf5239978ce23fc1769a8bc53f" exitCode=0
Jul 09 16:27:39.835418 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.835100    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz" event={"ID":"1915b720-a446-4114-b498-d96f1292a493","Type":"ContainerDied","Data":"315ce255581291f909e5e901eef5ce4b46902aaf5239978ce23fc1769a8bc53f"}
Jul 09 16:27:39.837264 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.837241    2586 generic.go:356] "Generic (PLEG): container finished" podID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerID="a491b432528f6ac9b469904b3d23eee9089d1ea7c41689d16580ddd800abbe08" exitCode=0
Jul 09 16:27:39.837367 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.837321    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc" event={"ID":"ad61d77a-3a6f-4545-b92d-6e86bf8176e0","Type":"ContainerDied","Data":"a491b432528f6ac9b469904b3d23eee9089d1ea7c41689d16580ddd800abbe08"}
Jul 09 16:27:39.839302 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.839287    2586 generic.go:356] "Generic (PLEG): container finished" podID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerID="8a412434db3075e23a609ccdda981e49481bec0f6afe2b56cd7733d2c205da51" exitCode=0
Jul 09 16:27:39.839400 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.839353    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6" event={"ID":"8e65f840-c5f3-4ee5-9901-db6192eb8577","Type":"ContainerDied","Data":"8a412434db3075e23a609ccdda981e49481bec0f6afe2b56cd7733d2c205da51"}
Jul 09 16:27:39.973968 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:39.973897    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:40.107992 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.107960    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-util\") pod \"4fd28db9-ca9f-47bc-b417-b822a2a653af\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") "
Jul 09 16:27:40.108167 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.108032    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkvnf\" (UniqueName: \"kubernetes.io/projected/4fd28db9-ca9f-47bc-b417-b822a2a653af-kube-api-access-mkvnf\") pod \"4fd28db9-ca9f-47bc-b417-b822a2a653af\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") "
Jul 09 16:27:40.108167 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.108055    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-bundle\") pod \"4fd28db9-ca9f-47bc-b417-b822a2a653af\" (UID: \"4fd28db9-ca9f-47bc-b417-b822a2a653af\") "
Jul 09 16:27:40.108483 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.108456    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-bundle" (OuterVolumeSpecName: "bundle") pod "4fd28db9-ca9f-47bc-b417-b822a2a653af" (UID: "4fd28db9-ca9f-47bc-b417-b822a2a653af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:40.110283 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.110259    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd28db9-ca9f-47bc-b417-b822a2a653af-kube-api-access-mkvnf" (OuterVolumeSpecName: "kube-api-access-mkvnf") pod "4fd28db9-ca9f-47bc-b417-b822a2a653af" (UID: "4fd28db9-ca9f-47bc-b417-b822a2a653af"). InnerVolumeSpecName "kube-api-access-mkvnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:27:40.113687 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.113662    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-util" (OuterVolumeSpecName: "util") pod "4fd28db9-ca9f-47bc-b417-b822a2a653af" (UID: "4fd28db9-ca9f-47bc-b417-b822a2a653af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:40.208851 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.208810    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkvnf\" (UniqueName: \"kubernetes.io/projected/4fd28db9-ca9f-47bc-b417-b822a2a653af-kube-api-access-mkvnf\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:40.208851 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.208843    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:40.208851 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.208857    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4fd28db9-ca9f-47bc-b417-b822a2a653af-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:40.845859 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.845829    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr"
Jul 09 16:27:40.846328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.845855    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr" event={"ID":"4fd28db9-ca9f-47bc-b417-b822a2a653af","Type":"ContainerDied","Data":"1421256a4ae073f8b8f2777d8935b97df945bfb840820c89010926e2abbec337"}
Jul 09 16:27:40.846328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.845905    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1421256a4ae073f8b8f2777d8935b97df945bfb840820c89010926e2abbec337"
Jul 09 16:27:40.848290 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.848269    2586 generic.go:356] "Generic (PLEG): container finished" podID="1915b720-a446-4114-b498-d96f1292a493" containerID="c2369c54aa41f51feb12610a59a4a6f53101d8099f9080239a1ee8a3b7fae8d7" exitCode=0
Jul 09 16:27:40.848406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.848343    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz" event={"ID":"1915b720-a446-4114-b498-d96f1292a493","Type":"ContainerDied","Data":"c2369c54aa41f51feb12610a59a4a6f53101d8099f9080239a1ee8a3b7fae8d7"}
Jul 09 16:27:40.850253 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.850234    2586 generic.go:356] "Generic (PLEG): container finished" podID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerID="6a902017e756cc393ea2a65d74ef09224c6c3dbad8c5a5c6cf51774f60b751b2" exitCode=0
Jul 09 16:27:40.850363 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.850321    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6" event={"ID":"8e65f840-c5f3-4ee5-9901-db6192eb8577","Type":"ContainerDied","Data":"6a902017e756cc393ea2a65d74ef09224c6c3dbad8c5a5c6cf51774f60b751b2"}
Jul 09 16:27:40.980864 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:40.980834    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:41.116455 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.116424    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljr7l\" (UniqueName: \"kubernetes.io/projected/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-kube-api-access-ljr7l\") pod \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") "
Jul 09 16:27:41.116606 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.116480    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-bundle\") pod \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") "
Jul 09 16:27:41.116606 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.116503    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-util\") pod \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\" (UID: \"ad61d77a-3a6f-4545-b92d-6e86bf8176e0\") "
Jul 09 16:27:41.117107 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.117077    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-bundle" (OuterVolumeSpecName: "bundle") pod "ad61d77a-3a6f-4545-b92d-6e86bf8176e0" (UID: "ad61d77a-3a6f-4545-b92d-6e86bf8176e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:41.118576 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.118546    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-kube-api-access-ljr7l" (OuterVolumeSpecName: "kube-api-access-ljr7l") pod "ad61d77a-3a6f-4545-b92d-6e86bf8176e0" (UID: "ad61d77a-3a6f-4545-b92d-6e86bf8176e0"). InnerVolumeSpecName "kube-api-access-ljr7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:27:41.121846 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.121825    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-util" (OuterVolumeSpecName: "util") pod "ad61d77a-3a6f-4545-b92d-6e86bf8176e0" (UID: "ad61d77a-3a6f-4545-b92d-6e86bf8176e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:41.217669 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.217636    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:41.217669 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.217665    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljr7l\" (UniqueName: \"kubernetes.io/projected/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-kube-api-access-ljr7l\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:41.217871 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.217677    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad61d77a-3a6f-4545-b92d-6e86bf8176e0-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:41.857369 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.857327    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc" event={"ID":"ad61d77a-3a6f-4545-b92d-6e86bf8176e0","Type":"ContainerDied","Data":"1a843f5551f7de3a830215d722172c75fe8caa0536793880f0b68c112198018b"}
Jul 09 16:27:41.857369 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.857359    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc"
Jul 09 16:27:41.857892 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.857385    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a843f5551f7de3a830215d722172c75fe8caa0536793880f0b68c112198018b"
Jul 09 16:27:41.993223 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:41.993197    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:42.020947 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.020917    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:42.125828 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.125738    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mwjz\" (UniqueName: \"kubernetes.io/projected/1915b720-a446-4114-b498-d96f1292a493-kube-api-access-8mwjz\") pod \"1915b720-a446-4114-b498-d96f1292a493\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") "
Jul 09 16:27:42.125828 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.125803    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc9tl\" (UniqueName: \"kubernetes.io/projected/8e65f840-c5f3-4ee5-9901-db6192eb8577-kube-api-access-mc9tl\") pod \"8e65f840-c5f3-4ee5-9901-db6192eb8577\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") "
Jul 09 16:27:42.125828 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.125831    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-bundle\") pod \"8e65f840-c5f3-4ee5-9901-db6192eb8577\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") "
Jul 09 16:27:42.126098 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.125878    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-util\") pod \"1915b720-a446-4114-b498-d96f1292a493\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") "
Jul 09 16:27:42.126098 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.125901    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-bundle\") pod \"1915b720-a446-4114-b498-d96f1292a493\" (UID: \"1915b720-a446-4114-b498-d96f1292a493\") "
Jul 09 16:27:42.126098 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.125946    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-util\") pod \"8e65f840-c5f3-4ee5-9901-db6192eb8577\" (UID: \"8e65f840-c5f3-4ee5-9901-db6192eb8577\") "
Jul 09 16:27:42.126533 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.126500    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-bundle" (OuterVolumeSpecName: "bundle") pod "8e65f840-c5f3-4ee5-9901-db6192eb8577" (UID: "8e65f840-c5f3-4ee5-9901-db6192eb8577"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:42.126631 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.126552    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-bundle" (OuterVolumeSpecName: "bundle") pod "1915b720-a446-4114-b498-d96f1292a493" (UID: "1915b720-a446-4114-b498-d96f1292a493"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:42.128022 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.127996    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e65f840-c5f3-4ee5-9901-db6192eb8577-kube-api-access-mc9tl" (OuterVolumeSpecName: "kube-api-access-mc9tl") pod "8e65f840-c5f3-4ee5-9901-db6192eb8577" (UID: "8e65f840-c5f3-4ee5-9901-db6192eb8577"). InnerVolumeSpecName "kube-api-access-mc9tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:27:42.128100 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.128085    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1915b720-a446-4114-b498-d96f1292a493-kube-api-access-8mwjz" (OuterVolumeSpecName: "kube-api-access-8mwjz") pod "1915b720-a446-4114-b498-d96f1292a493" (UID: "1915b720-a446-4114-b498-d96f1292a493"). InnerVolumeSpecName "kube-api-access-8mwjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:27:42.134896 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.134865    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-util" (OuterVolumeSpecName: "util") pod "8e65f840-c5f3-4ee5-9901-db6192eb8577" (UID: "8e65f840-c5f3-4ee5-9901-db6192eb8577"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:42.135050 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.135032    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-util" (OuterVolumeSpecName: "util") pod "1915b720-a446-4114-b498-d96f1292a493" (UID: "1915b720-a446-4114-b498-d96f1292a493"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:27:42.227207 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.227158    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mwjz\" (UniqueName: \"kubernetes.io/projected/1915b720-a446-4114-b498-d96f1292a493-kube-api-access-8mwjz\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:42.227207 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.227199    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc9tl\" (UniqueName: \"kubernetes.io/projected/8e65f840-c5f3-4ee5-9901-db6192eb8577-kube-api-access-mc9tl\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:42.227207 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.227210    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:42.227207 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.227218    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:42.227207 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.227226    2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1915b720-a446-4114-b498-d96f1292a493-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:42.227467 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.227233    2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8e65f840-c5f3-4ee5-9901-db6192eb8577-util\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:42.863366 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.863320    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz" event={"ID":"1915b720-a446-4114-b498-d96f1292a493","Type":"ContainerDied","Data":"49e2c608cc28adbbf57dbe249811513a4a225d599bb42677f6ec819ce21cb61f"}
Jul 09 16:27:42.863803 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.863373    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49e2c608cc28adbbf57dbe249811513a4a225d599bb42677f6ec819ce21cb61f"
Jul 09 16:27:42.863803 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.863333    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz"
Jul 09 16:27:42.865539 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.865373    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6"
Jul 09 16:27:42.865539 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.865386    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6" event={"ID":"8e65f840-c5f3-4ee5-9901-db6192eb8577","Type":"ContainerDied","Data":"97a3d7e33c57b30af8297c154ee8fdce32e115dd8c000edb2663611a156d7c01"}
Jul 09 16:27:42.865539 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:42.865424    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97a3d7e33c57b30af8297c154ee8fdce32e115dd8c000edb2663611a156d7c01"
Jul 09 16:27:51.749114 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:51.749050    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75d5454df9-tsm28" podUID="8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" containerName="console" containerID="cri-o://080c95c696e21a18a09c282e0ed966bfee04d3f911a23bb4f7faf56ca2f53a8b" gracePeriod=15
Jul 09 16:27:51.910315 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:51.910279    2586 generic.go:356] "Generic (PLEG): container finished" podID="8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" containerID="080c95c696e21a18a09c282e0ed966bfee04d3f911a23bb4f7faf56ca2f53a8b" exitCode=0
Jul 09 16:27:51.910459 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:51.910343    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d5454df9-tsm28" event={"ID":"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc","Type":"ContainerDied","Data":"080c95c696e21a18a09c282e0ed966bfee04d3f911a23bb4f7faf56ca2f53a8b"}
Jul 09 16:27:51.993076 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:51.993049    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:27:52.116345 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116313    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-oauth-serving-cert\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116370    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-trusted-ca-bundle\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116393    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-service-ca\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116424    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-oauth-config\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116466    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-serving-cert\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116490    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gw8s\" (UniqueName: \"kubernetes.io/projected/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-kube-api-access-6gw8s\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116524 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116516    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-config\") pod \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\" (UID: \"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc\") "
Jul 09 16:27:52.116913 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116881    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:27:52.117022 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.116995    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-config" (OuterVolumeSpecName: "console-config") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:27:52.117022 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.117007    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-service-ca" (OuterVolumeSpecName: "service-ca") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:27:52.117124 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.117070    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:27:52.118658 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.118634    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:27:52.118770 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.118729    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:27:52.118770 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.118734    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-kube-api-access-6gw8s" (OuterVolumeSpecName: "kube-api-access-6gw8s") pod "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" (UID: "8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc"). InnerVolumeSpecName "kube-api-access-6gw8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:27:52.218081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218041    2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.218081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218072    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gw8s\" (UniqueName: \"kubernetes.io/projected/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-kube-api-access-6gw8s\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.218081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218083    2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.218297 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218094    2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-oauth-serving-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.218297 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218102    2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-trusted-ca-bundle\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.218297 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218111    2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.218297 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.218119    2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc-console-oauth-config\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:27:52.917129 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.917103    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75d5454df9-tsm28"
Jul 09 16:27:52.917129 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.917119    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75d5454df9-tsm28" event={"ID":"8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc","Type":"ContainerDied","Data":"189aaf3d03531831c5bb6de490dddcb45e76c0b3ee647aa0c11786989afbac9b"}
Jul 09 16:27:52.917620 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.917174    2586 scope.go:117] "RemoveContainer" containerID="080c95c696e21a18a09c282e0ed966bfee04d3f911a23bb4f7faf56ca2f53a8b"
Jul 09 16:27:52.940563 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.940531    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75d5454df9-tsm28"]
Jul 09 16:27:52.947095 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:52.947068    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75d5454df9-tsm28"]
Jul 09 16:27:54.771913 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:27:54.771878    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" path="/var/lib/kubelet/pods/8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc/volumes"
Jul 09 16:28:03.107349 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107307    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"]
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107634    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107645    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107652    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107657    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107667    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107673    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107679    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107684    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107697    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107702    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107709    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="pull"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107713    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="pull"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107720    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107725    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107733    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" containerName="console"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107737    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" containerName="console"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107747    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107753    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="util"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107772    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107777    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="extract"
Jul 09 16:28:03.107743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107782    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="pull"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107787    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="pull"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107793    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="pull"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107800    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="pull"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107808    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="pull"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107815    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="pull"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107861    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e65f840-c5f3-4ee5-9901-db6192eb8577" containerName="extract"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107869    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad4ddc2-a39a-46c8-a3ab-3bf7f04e25fc" containerName="console"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107875    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1915b720-a446-4114-b498-d96f1292a493" containerName="extract"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107884    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd28db9-ca9f-47bc-b417-b822a2a653af" containerName="extract"
Jul 09 16:28:03.108370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.107889    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad61d77a-3a6f-4545-b92d-6e86bf8176e0" containerName="extract"
Jul 09 16:28:03.112059 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.112039    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.114957 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.114932    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n7pqw\""
Jul 09 16:28:03.115058 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.114962    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 09 16:28:03.115733 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.115720    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 16:28:03.115796 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.115734    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 09 16:28:03.115830 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.115799    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 16:28:03.127379 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.127352    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"]
Jul 09 16:28:03.214205 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.214172    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.214365 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.214289    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn2ck\" (UniqueName: \"kubernetes.io/projected/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-kube-api-access-nn2ck\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.214365 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.214336    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.315064 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.315029    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn2ck\" (UniqueName: \"kubernetes.io/projected/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-kube-api-access-nn2ck\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.315064 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.315067    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.315263 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.315107    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.315832 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.315814    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.317560 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.317528    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.329876 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.329843    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn2ck\" (UniqueName: \"kubernetes.io/projected/662e7b88-5a52-40ff-8cd9-7ee9acb9f830-kube-api-access-nn2ck\") pod \"kuadrant-console-plugin-66b5774985-lgqqs\" (UID: \"662e7b88-5a52-40ff-8cd9-7ee9acb9f830\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.420963 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.420883    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"
Jul 09 16:28:03.560370 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.560345    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs"]
Jul 09 16:28:03.562073 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:28:03.562046    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod662e7b88_5a52_40ff_8cd9_7ee9acb9f830.slice/crio-21ad172d1de94e673e6e5d7b0d74b4ffb2d823340574be711f2e3b044c342b1b WatchSource:0}: Error finding container 21ad172d1de94e673e6e5d7b0d74b4ffb2d823340574be711f2e3b044c342b1b: Status 404 returned error can't find the container with id 21ad172d1de94e673e6e5d7b0d74b4ffb2d823340574be711f2e3b044c342b1b
Jul 09 16:28:03.975565 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:03.975533    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs" event={"ID":"662e7b88-5a52-40ff-8cd9-7ee9acb9f830","Type":"ContainerStarted","Data":"21ad172d1de94e673e6e5d7b0d74b4ffb2d823340574be711f2e3b044c342b1b"}
Jul 09 16:28:27.112480 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:27.112447    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs" event={"ID":"662e7b88-5a52-40ff-8cd9-7ee9acb9f830","Type":"ContainerStarted","Data":"e73903eecc6212b6c2a129382f8f8528192af3c1ec48487e9cdad1a541e48588"}
Jul 09 16:28:27.144130 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:27.144076    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lgqqs" podStartSLOduration=1.253381162 podStartE2EDuration="24.144056297s" totalImagesPullingTime="22.890675135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:28:27.140998444 +0000 UTC m=+518.066952723" watchObservedRunningTime="2026-07-09 16:28:27.144056297 +0000 UTC m=+518.070010575"
Jul 09 16:28:45.568319 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.568224    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:28:45.570743 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.570718    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.573167 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.573143    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 16:28:45.580049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.580026    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:28:45.613909 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.613882    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:28:45.695375 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.695340    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f86287e-03ef-4445-a435-4c97ce49b560-config-file\") pod \"limitador-limitador-54d6d988d5-cplnw\" (UID: \"8f86287e-03ef-4445-a435-4c97ce49b560\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.695546 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.695407    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwbth\" (UniqueName: \"kubernetes.io/projected/8f86287e-03ef-4445-a435-4c97ce49b560-kube-api-access-xwbth\") pod \"limitador-limitador-54d6d988d5-cplnw\" (UID: \"8f86287e-03ef-4445-a435-4c97ce49b560\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.796310 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.796263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f86287e-03ef-4445-a435-4c97ce49b560-config-file\") pod \"limitador-limitador-54d6d988d5-cplnw\" (UID: \"8f86287e-03ef-4445-a435-4c97ce49b560\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.796480 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.796333    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwbth\" (UniqueName: \"kubernetes.io/projected/8f86287e-03ef-4445-a435-4c97ce49b560-kube-api-access-xwbth\") pod \"limitador-limitador-54d6d988d5-cplnw\" (UID: \"8f86287e-03ef-4445-a435-4c97ce49b560\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.797031 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.797011    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8f86287e-03ef-4445-a435-4c97ce49b560-config-file\") pod \"limitador-limitador-54d6d988d5-cplnw\" (UID: \"8f86287e-03ef-4445-a435-4c97ce49b560\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.805375 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.805346    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwbth\" (UniqueName: \"kubernetes.io/projected/8f86287e-03ef-4445-a435-4c97ce49b560-kube-api-access-xwbth\") pod \"limitador-limitador-54d6d988d5-cplnw\" (UID: \"8f86287e-03ef-4445-a435-4c97ce49b560\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:45.881253 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:45.881212    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:46.017312 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.017283    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:28:46.019644 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:28:46.019584    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f86287e_03ef_4445_a435_4c97ce49b560.slice/crio-f7cf1c6263541c1438d0e6ddf05b6da5b23b4936c976920eb7887593d4743ac0 WatchSource:0}: Error finding container f7cf1c6263541c1438d0e6ddf05b6da5b23b4936c976920eb7887593d4743ac0: Status 404 returned error can't find the container with id f7cf1c6263541c1438d0e6ddf05b6da5b23b4936c976920eb7887593d4743ac0
Jul 09 16:28:46.040912 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.040885    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-s8zrb"]
Jul 09 16:28:46.062295 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.062252    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s8zrb"]
Jul 09 16:28:46.062456 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.062352    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:46.065734 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.065714    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5mhv5\""
Jul 09 16:28:46.199277 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.199198    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf9d9\" (UniqueName: \"kubernetes.io/projected/600dc0c0-9951-420b-bd2a-745fd7306aeb-kube-api-access-kf9d9\") pod \"authorino-58cd955675-s8zrb\" (UID: \"600dc0c0-9951-420b-bd2a-745fd7306aeb\") " pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:46.207002 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.206972    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw" event={"ID":"8f86287e-03ef-4445-a435-4c97ce49b560","Type":"ContainerStarted","Data":"f7cf1c6263541c1438d0e6ddf05b6da5b23b4936c976920eb7887593d4743ac0"}
Jul 09 16:28:46.299876 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.299835    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf9d9\" (UniqueName: \"kubernetes.io/projected/600dc0c0-9951-420b-bd2a-745fd7306aeb-kube-api-access-kf9d9\") pod \"authorino-58cd955675-s8zrb\" (UID: \"600dc0c0-9951-420b-bd2a-745fd7306aeb\") " pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:46.308521 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.308497    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf9d9\" (UniqueName: \"kubernetes.io/projected/600dc0c0-9951-420b-bd2a-745fd7306aeb-kube-api-access-kf9d9\") pod \"authorino-58cd955675-s8zrb\" (UID: \"600dc0c0-9951-420b-bd2a-745fd7306aeb\") " pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:46.372189 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.372156    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:46.536791 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:46.536741    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s8zrb"]
Jul 09 16:28:46.540058 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:28:46.540032    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod600dc0c0_9951_420b_bd2a_745fd7306aeb.slice/crio-6df59183152feebcfdf4a6c978efe070e3eda06591bc7fffed9cf6d3ca1e9a7c WatchSource:0}: Error finding container 6df59183152feebcfdf4a6c978efe070e3eda06591bc7fffed9cf6d3ca1e9a7c: Status 404 returned error can't find the container with id 6df59183152feebcfdf4a6c978efe070e3eda06591bc7fffed9cf6d3ca1e9a7c
Jul 09 16:28:47.213335 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:47.213298    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s8zrb" event={"ID":"600dc0c0-9951-420b-bd2a-745fd7306aeb","Type":"ContainerStarted","Data":"6df59183152feebcfdf4a6c978efe070e3eda06591bc7fffed9cf6d3ca1e9a7c"}
Jul 09 16:28:51.760118 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:51.760086    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s8zrb"]
Jul 09 16:28:53.252812 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:53.252774    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s8zrb" event={"ID":"600dc0c0-9951-420b-bd2a-745fd7306aeb","Type":"ContainerStarted","Data":"e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439"}
Jul 09 16:28:53.253267 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:53.252821    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-s8zrb" podUID="600dc0c0-9951-420b-bd2a-745fd7306aeb" containerName="authorino" containerID="cri-o://e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439" gracePeriod=30
Jul 09 16:28:53.254566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:53.254523    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw" event={"ID":"8f86287e-03ef-4445-a435-4c97ce49b560","Type":"ContainerStarted","Data":"6a49001a145d9e0869d679963da630c1f0f3da2ac83624df8f2ba16d7fe54b88"}
Jul 09 16:28:53.254712 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:53.254611    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:28:53.268777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:53.268715    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-s8zrb" podStartSLOduration=1.204125986 podStartE2EDuration="7.268697033s" totalImagesPullingTime="6.064571047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:28:53.266792817 +0000 UTC m=+544.192747089" watchObservedRunningTime="2026-07-09 16:28:53.268697033 +0000 UTC m=+544.194651310"
Jul 09 16:28:53.283851 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:53.283783    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw" podStartSLOduration=1.710364036 podStartE2EDuration="8.28375271s" totalImagesPullingTime="6.573388674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:28:53.282478193 +0000 UTC m=+544.208432466" watchObservedRunningTime="2026-07-09 16:28:53.28375271 +0000 UTC m=+544.209706995"
Jul 09 16:28:54.112418 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.112380    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:54.261266 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.261170    2586 generic.go:356] "Generic (PLEG): container finished" podID="600dc0c0-9951-420b-bd2a-745fd7306aeb" containerID="e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439" exitCode=0
Jul 09 16:28:54.261266 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.261208    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s8zrb"
Jul 09 16:28:54.261266 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.261263    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s8zrb" event={"ID":"600dc0c0-9951-420b-bd2a-745fd7306aeb","Type":"ContainerDied","Data":"e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439"}
Jul 09 16:28:54.261792 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.261298    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s8zrb" event={"ID":"600dc0c0-9951-420b-bd2a-745fd7306aeb","Type":"ContainerDied","Data":"6df59183152feebcfdf4a6c978efe070e3eda06591bc7fffed9cf6d3ca1e9a7c"}
Jul 09 16:28:54.261792 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.261317    2586 scope.go:117] "RemoveContainer" containerID="e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439"
Jul 09 16:28:54.269728 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.269698    2586 scope.go:117] "RemoveContainer" containerID="e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439"
Jul 09 16:28:54.270024 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:28:54.270005    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439\": container with ID starting with e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439 not found: ID does not exist" containerID="e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439"
Jul 09 16:28:54.270086 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.270033    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439"} err="failed to get container status \"e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439\": rpc error: code = NotFound desc = could not find container \"e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439\": container with ID starting with e1365d7069eab133814166e80e199a1353b43c9418ad7b3c06c3d84ebec2c439 not found: ID does not exist"
Jul 09 16:28:54.274283 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.274268    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf9d9\" (UniqueName: \"kubernetes.io/projected/600dc0c0-9951-420b-bd2a-745fd7306aeb-kube-api-access-kf9d9\") pod \"600dc0c0-9951-420b-bd2a-745fd7306aeb\" (UID: \"600dc0c0-9951-420b-bd2a-745fd7306aeb\") "
Jul 09 16:28:54.276324 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.276297    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600dc0c0-9951-420b-bd2a-745fd7306aeb-kube-api-access-kf9d9" (OuterVolumeSpecName: "kube-api-access-kf9d9") pod "600dc0c0-9951-420b-bd2a-745fd7306aeb" (UID: "600dc0c0-9951-420b-bd2a-745fd7306aeb"). InnerVolumeSpecName "kube-api-access-kf9d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:28:54.375497 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.375453    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf9d9\" (UniqueName: \"kubernetes.io/projected/600dc0c0-9951-420b-bd2a-745fd7306aeb-kube-api-access-kf9d9\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:28:54.585926 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.585899    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s8zrb"]
Jul 09 16:28:54.590363 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.590340    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-s8zrb"]
Jul 09 16:28:54.772179 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:28:54.772142    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600dc0c0-9951-420b-bd2a-745fd7306aeb" path="/var/lib/kubelet/pods/600dc0c0-9951-420b-bd2a-745fd7306aeb/volumes"
Jul 09 16:29:04.264116 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:04.264083    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cplnw"
Jul 09 16:29:19.409834 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.409798    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2swgg"]
Jul 09 16:29:19.410260 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.410129    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600dc0c0-9951-420b-bd2a-745fd7306aeb" containerName="authorino"
Jul 09 16:29:19.410260 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.410139    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="600dc0c0-9951-420b-bd2a-745fd7306aeb" containerName="authorino"
Jul 09 16:29:19.410260 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.410204    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="600dc0c0-9951-420b-bd2a-745fd7306aeb" containerName="authorino"
Jul 09 16:29:19.413058 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.413043    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:19.415386 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.415356    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5mhv5\""
Jul 09 16:29:19.420970 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.420936    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2swgg"]
Jul 09 16:29:19.494905 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.494866    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvplr\" (UniqueName: \"kubernetes.io/projected/41572422-2c06-47a1-baaf-0e283bd01d62-kube-api-access-tvplr\") pod \"authorino-5d4b67c446-2swgg\" (UID: \"41572422-2c06-47a1-baaf-0e283bd01d62\") " pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:19.595535 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.595479    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvplr\" (UniqueName: \"kubernetes.io/projected/41572422-2c06-47a1-baaf-0e283bd01d62-kube-api-access-tvplr\") pod \"authorino-5d4b67c446-2swgg\" (UID: \"41572422-2c06-47a1-baaf-0e283bd01d62\") " pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:19.605439 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.605404    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvplr\" (UniqueName: \"kubernetes.io/projected/41572422-2c06-47a1-baaf-0e283bd01d62-kube-api-access-tvplr\") pod \"authorino-5d4b67c446-2swgg\" (UID: \"41572422-2c06-47a1-baaf-0e283bd01d62\") " pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:19.632067 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.632037    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2swgg"]
Jul 09 16:29:19.632318 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.632302    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:19.765827 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.765736    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2swgg"]
Jul 09 16:29:19.768027 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:29:19.767998    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41572422_2c06_47a1_baaf_0e283bd01d62.slice/crio-049eb7123a72e0894ed0657d039abf9b6f8a38c585a048eab0c12c0d6ce8ede1 WatchSource:0}: Error finding container 049eb7123a72e0894ed0657d039abf9b6f8a38c585a048eab0c12c0d6ce8ede1: Status 404 returned error can't find the container with id 049eb7123a72e0894ed0657d039abf9b6f8a38c585a048eab0c12c0d6ce8ede1
Jul 09 16:29:19.774852 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.774817    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5896f8fb9f-7j7lv"]
Jul 09 16:29:19.779212 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.779193    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:19.781552 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.781532    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 16:29:19.786225 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.786206    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5896f8fb9f-7j7lv"]
Jul 09 16:29:19.897605 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.897572    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwjhb\" (UniqueName: \"kubernetes.io/projected/16d1b4e7-a004-4f82-b0c1-f129769690ed-kube-api-access-jwjhb\") pod \"authorino-5896f8fb9f-7j7lv\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") " pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:19.897821 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.897636    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16d1b4e7-a004-4f82-b0c1-f129769690ed-tls-cert\") pod \"authorino-5896f8fb9f-7j7lv\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") " pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:19.998973 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.998879    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16d1b4e7-a004-4f82-b0c1-f129769690ed-tls-cert\") pod \"authorino-5896f8fb9f-7j7lv\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") " pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:19.999138 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:19.998983    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwjhb\" (UniqueName: \"kubernetes.io/projected/16d1b4e7-a004-4f82-b0c1-f129769690ed-kube-api-access-jwjhb\") pod \"authorino-5896f8fb9f-7j7lv\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") " pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:20.001315 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.001296    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16d1b4e7-a004-4f82-b0c1-f129769690ed-tls-cert\") pod \"authorino-5896f8fb9f-7j7lv\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") " pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:20.009918 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.009889    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwjhb\" (UniqueName: \"kubernetes.io/projected/16d1b4e7-a004-4f82-b0c1-f129769690ed-kube-api-access-jwjhb\") pod \"authorino-5896f8fb9f-7j7lv\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") " pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:20.090284 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.090251    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:29:20.240131 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.240108    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5896f8fb9f-7j7lv"]
Jul 09 16:29:20.242870 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:29:20.242832    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16d1b4e7_a004_4f82_b0c1_f129769690ed.slice/crio-0fbcde281ff6c7e7f6365d52a52bb598ea200e8a689df062f2977242962ab409 WatchSource:0}: Error finding container 0fbcde281ff6c7e7f6365d52a52bb598ea200e8a689df062f2977242962ab409: Status 404 returned error can't find the container with id 0fbcde281ff6c7e7f6365d52a52bb598ea200e8a689df062f2977242962ab409
Jul 09 16:29:20.392726 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.392689    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv" event={"ID":"16d1b4e7-a004-4f82-b0c1-f129769690ed","Type":"ContainerStarted","Data":"0fbcde281ff6c7e7f6365d52a52bb598ea200e8a689df062f2977242962ab409"}
Jul 09 16:29:20.394111 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.394088    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2swgg" event={"ID":"41572422-2c06-47a1-baaf-0e283bd01d62","Type":"ContainerStarted","Data":"78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b"}
Jul 09 16:29:20.394236 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.394122    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2swgg" event={"ID":"41572422-2c06-47a1-baaf-0e283bd01d62","Type":"ContainerStarted","Data":"049eb7123a72e0894ed0657d039abf9b6f8a38c585a048eab0c12c0d6ce8ede1"}
Jul 09 16:29:20.394236 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.394166    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-2swgg" podUID="41572422-2c06-47a1-baaf-0e283bd01d62" containerName="authorino" containerID="cri-o://78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b" gracePeriod=30
Jul 09 16:29:20.410801 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.410742    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-2swgg" podStartSLOduration=1.050360092 podStartE2EDuration="1.410725375s" totalImagesPullingTime="360.365283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:29:20.409678472 +0000 UTC m=+571.335632752" watchObservedRunningTime="2026-07-09 16:29:20.410725375 +0000 UTC m=+571.336679652"
Jul 09 16:29:20.644364 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.644303    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:20.704722 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.704689    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvplr\" (UniqueName: \"kubernetes.io/projected/41572422-2c06-47a1-baaf-0e283bd01d62-kube-api-access-tvplr\") pod \"41572422-2c06-47a1-baaf-0e283bd01d62\" (UID: \"41572422-2c06-47a1-baaf-0e283bd01d62\") "
Jul 09 16:29:20.706961 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.706934    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41572422-2c06-47a1-baaf-0e283bd01d62-kube-api-access-tvplr" (OuterVolumeSpecName: "kube-api-access-tvplr") pod "41572422-2c06-47a1-baaf-0e283bd01d62" (UID: "41572422-2c06-47a1-baaf-0e283bd01d62"). InnerVolumeSpecName "kube-api-access-tvplr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:29:20.805300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:20.805269    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvplr\" (UniqueName: \"kubernetes.io/projected/41572422-2c06-47a1-baaf-0e283bd01d62-kube-api-access-tvplr\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:29:21.400156 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.400121    2586 generic.go:356] "Generic (PLEG): container finished" podID="41572422-2c06-47a1-baaf-0e283bd01d62" containerID="78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b" exitCode=0
Jul 09 16:29:21.400356 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.400178    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-2swgg"
Jul 09 16:29:21.400356 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.400204    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2swgg" event={"ID":"41572422-2c06-47a1-baaf-0e283bd01d62","Type":"ContainerDied","Data":"78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b"}
Jul 09 16:29:21.400356 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.400242    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-2swgg" event={"ID":"41572422-2c06-47a1-baaf-0e283bd01d62","Type":"ContainerDied","Data":"049eb7123a72e0894ed0657d039abf9b6f8a38c585a048eab0c12c0d6ce8ede1"}
Jul 09 16:29:21.400356 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.400265    2586 scope.go:117] "RemoveContainer" containerID="78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b"
Jul 09 16:29:21.401985 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.401959    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv" event={"ID":"16d1b4e7-a004-4f82-b0c1-f129769690ed","Type":"ContainerStarted","Data":"85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e"}
Jul 09 16:29:21.409102 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.409083    2586 scope.go:117] "RemoveContainer" containerID="78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b"
Jul 09 16:29:21.409351 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:29:21.409333    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b\": container with ID starting with 78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b not found: ID does not exist" containerID="78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b"
Jul 09 16:29:21.409439 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.409361    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b"} err="failed to get container status \"78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b\": rpc error: code = NotFound desc = could not find container \"78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b\": container with ID starting with 78b3d5695a435b7c7b829c8342dff51be16d963049a05db41eb7296097dbb51b not found: ID does not exist"
Jul 09 16:29:21.421692 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.421640    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv" podStartSLOduration=1.900069068 podStartE2EDuration="2.421625446s" totalImagesPullingTime="521.556378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:29:21.419215217 +0000 UTC m=+572.345169508" watchObservedRunningTime="2026-07-09 16:29:21.421625446 +0000 UTC m=+572.347579725"
Jul 09 16:29:21.435154 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.435128    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2swgg"]
Jul 09 16:29:21.440465 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:21.440431    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-2swgg"]
Jul 09 16:29:22.772877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:22.772843    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41572422-2c06-47a1-baaf-0e283bd01d62" path="/var/lib/kubelet/pods/41572422-2c06-47a1-baaf-0e283bd01d62/volumes"
Jul 09 16:29:50.645016 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:50.644986    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:29:50.646112 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:29:50.646089    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:30:02.987565 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:30:02.987518    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:30:24.927295 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:30:24.927206    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:30:39.832709 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:30:39.832666    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:30:50.431226 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:30:50.431175    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:31:22.139809 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:22.139769    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:31:29.730952 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:29.730917    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:31:31.707840 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.707800    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"]
Jul 09 16:31:31.708343 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.708323    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41572422-2c06-47a1-baaf-0e283bd01d62" containerName="authorino"
Jul 09 16:31:31.708417 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.708346    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="41572422-2c06-47a1-baaf-0e283bd01d62" containerName="authorino"
Jul 09 16:31:31.708467 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.708431    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="41572422-2c06-47a1-baaf-0e283bd01d62" containerName="authorino"
Jul 09 16:31:31.711274 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.711253    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.713720 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.713695    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 16:31:31.714630 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.714609    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 16:31:31.714630 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.714621    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2dlbw\""
Jul 09 16:31:31.714802 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.714646    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 09 16:31:31.719421 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.719395    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"]
Jul 09 16:31:31.830140 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.830105    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgsb7\" (UniqueName: \"kubernetes.io/projected/8f361d48-a40f-4706-90a8-13fca4e7b8b2-kube-api-access-bgsb7\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.830302 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.830147    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.830302 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.830172    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.830302 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.830255    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.830302 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.830288    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f361d48-a40f-4706-90a8-13fca4e7b8b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.830443 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.830318    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931234 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931194    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931415 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931257    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931415 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931281    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f361d48-a40f-4706-90a8-13fca4e7b8b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931415 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931312    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931415 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931391    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgsb7\" (UniqueName: \"kubernetes.io/projected/8f361d48-a40f-4706-90a8-13fca4e7b8b2-kube-api-access-bgsb7\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931634 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931420    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931692 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931637    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931746 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931697    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.931857 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.931840    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.933554 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.933531    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f361d48-a40f-4706-90a8-13fca4e7b8b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.933846 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.933830    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f361d48-a40f-4706-90a8-13fca4e7b8b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:31.940880 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:31.940853    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgsb7\" (UniqueName: \"kubernetes.io/projected/8f361d48-a40f-4706-90a8-13fca4e7b8b2-kube-api-access-bgsb7\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5\" (UID: \"8f361d48-a40f-4706-90a8-13fca4e7b8b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:32.021596 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:32.021507    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:32.160093 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:32.160010    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"]
Jul 09 16:31:32.162420 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:31:32.162394    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f361d48_a40f_4706_90a8_13fca4e7b8b2.slice/crio-61faeb85e280ec1778500900ab7726664b07fece3bbc04a5350fe0b9cb7c4e0d WatchSource:0}: Error finding container 61faeb85e280ec1778500900ab7726664b07fece3bbc04a5350fe0b9cb7c4e0d: Status 404 returned error can't find the container with id 61faeb85e280ec1778500900ab7726664b07fece3bbc04a5350fe0b9cb7c4e0d
Jul 09 16:31:32.163819 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:32.163801    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 16:31:33.068713 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:33.068670    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5" event={"ID":"8f361d48-a40f-4706-90a8-13fca4e7b8b2","Type":"ContainerStarted","Data":"61faeb85e280ec1778500900ab7726664b07fece3bbc04a5350fe0b9cb7c4e0d"}
Jul 09 16:31:34.079993 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:34.079950    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5" event={"ID":"8f361d48-a40f-4706-90a8-13fca4e7b8b2","Type":"ContainerStarted","Data":"ec0333046e6ba063c949575d7c3c9736994db3ba0615f3b0880431116ff00bd9"}
Jul 09 16:31:34.108609 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:34.108541    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5" podStartSLOduration=2.093006018 podStartE2EDuration="3.108519519s" totalImagesPullingTime="1.015513501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:31:34.106519001 +0000 UTC m=+705.032473281" watchObservedRunningTime="2026-07-09 16:31:34.108519519 +0000 UTC m=+705.034473796"
Jul 09 16:31:39.730068 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:39.730023    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:31:42.022016 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:42.021914    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:42.022016 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:42.022004    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:42.034833 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:42.034806    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:31:42.132005 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:31:42.131978    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5"
Jul 09 16:32:15.663127 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.663090    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-ffd45bb66-4spxv"]
Jul 09 16:32:15.666317 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.666299    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.675533 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.675509    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ffd45bb66-4spxv"]
Jul 09 16:32:15.796820 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.796792    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnplq\" (UniqueName: \"kubernetes.io/projected/b3aae518-6657-413e-8c5e-cd9719e6bb7a-kube-api-access-pnplq\") pod \"authorino-ffd45bb66-4spxv\" (UID: \"b3aae518-6657-413e-8c5e-cd9719e6bb7a\") " pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.796993 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.796867    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b3aae518-6657-413e-8c5e-cd9719e6bb7a-tls-cert\") pod \"authorino-ffd45bb66-4spxv\" (UID: \"b3aae518-6657-413e-8c5e-cd9719e6bb7a\") " pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.898246 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.898202    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b3aae518-6657-413e-8c5e-cd9719e6bb7a-tls-cert\") pod \"authorino-ffd45bb66-4spxv\" (UID: \"b3aae518-6657-413e-8c5e-cd9719e6bb7a\") " pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.898458 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.898379    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnplq\" (UniqueName: \"kubernetes.io/projected/b3aae518-6657-413e-8c5e-cd9719e6bb7a-kube-api-access-pnplq\") pod \"authorino-ffd45bb66-4spxv\" (UID: \"b3aae518-6657-413e-8c5e-cd9719e6bb7a\") " pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.900644 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.900619    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b3aae518-6657-413e-8c5e-cd9719e6bb7a-tls-cert\") pod \"authorino-ffd45bb66-4spxv\" (UID: \"b3aae518-6657-413e-8c5e-cd9719e6bb7a\") " pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.907504 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.907479    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnplq\" (UniqueName: \"kubernetes.io/projected/b3aae518-6657-413e-8c5e-cd9719e6bb7a-kube-api-access-pnplq\") pod \"authorino-ffd45bb66-4spxv\" (UID: \"b3aae518-6657-413e-8c5e-cd9719e6bb7a\") " pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:15.975419 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:15.975340    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-ffd45bb66-4spxv"
Jul 09 16:32:16.120842 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:16.120814    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-ffd45bb66-4spxv"]
Jul 09 16:32:16.122477 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:32:16.122443    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3aae518_6657_413e_8c5e_cd9719e6bb7a.slice/crio-307c1ca781a44050407247021f9a4527c23840a022eff3131d50960bf96119bd WatchSource:0}: Error finding container 307c1ca781a44050407247021f9a4527c23840a022eff3131d50960bf96119bd: Status 404 returned error can't find the container with id 307c1ca781a44050407247021f9a4527c23840a022eff3131d50960bf96119bd
Jul 09 16:32:16.291681 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:16.291587    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ffd45bb66-4spxv" event={"ID":"b3aae518-6657-413e-8c5e-cd9719e6bb7a","Type":"ContainerStarted","Data":"307c1ca781a44050407247021f9a4527c23840a022eff3131d50960bf96119bd"}
Jul 09 16:32:17.299951 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.299866    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-ffd45bb66-4spxv" event={"ID":"b3aae518-6657-413e-8c5e-cd9719e6bb7a","Type":"ContainerStarted","Data":"2c9bf71567ee15a98b1f8cffad0679a46918d78c9b9a513d2a329a5908151742"}
Jul 09 16:32:17.328130 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.328066    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-ffd45bb66-4spxv" podStartSLOduration=1.606124122 podStartE2EDuration="2.328044955s" totalImagesPullingTime="721.920833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:32:17.328019492 +0000 UTC m=+748.253973770" watchObservedRunningTime="2026-07-09 16:32:17.328044955 +0000 UTC m=+748.253999235"
Jul 09 16:32:17.401736 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.401700    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5896f8fb9f-7j7lv"]
Jul 09 16:32:17.401985 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.401952    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv" podUID="16d1b4e7-a004-4f82-b0c1-f129769690ed" containerName="authorino" containerID="cri-o://85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e" gracePeriod=30
Jul 09 16:32:17.662107 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.662079    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:32:17.721219 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.721183    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16d1b4e7-a004-4f82-b0c1-f129769690ed-tls-cert\") pod \"16d1b4e7-a004-4f82-b0c1-f129769690ed\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") "
Jul 09 16:32:17.721402 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.721290    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwjhb\" (UniqueName: \"kubernetes.io/projected/16d1b4e7-a004-4f82-b0c1-f129769690ed-kube-api-access-jwjhb\") pod \"16d1b4e7-a004-4f82-b0c1-f129769690ed\" (UID: \"16d1b4e7-a004-4f82-b0c1-f129769690ed\") "
Jul 09 16:32:17.723322 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.723291    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d1b4e7-a004-4f82-b0c1-f129769690ed-kube-api-access-jwjhb" (OuterVolumeSpecName: "kube-api-access-jwjhb") pod "16d1b4e7-a004-4f82-b0c1-f129769690ed" (UID: "16d1b4e7-a004-4f82-b0c1-f129769690ed"). InnerVolumeSpecName "kube-api-access-jwjhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:32:17.731986 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.731959    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16d1b4e7-a004-4f82-b0c1-f129769690ed-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "16d1b4e7-a004-4f82-b0c1-f129769690ed" (UID: "16d1b4e7-a004-4f82-b0c1-f129769690ed"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:32:17.822068 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.822031    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwjhb\" (UniqueName: \"kubernetes.io/projected/16d1b4e7-a004-4f82-b0c1-f129769690ed-kube-api-access-jwjhb\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:32:17.822068 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:17.822062    2586 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16d1b4e7-a004-4f82-b0c1-f129769690ed-tls-cert\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:32:18.306164 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.306078    2586 generic.go:356] "Generic (PLEG): container finished" podID="16d1b4e7-a004-4f82-b0c1-f129769690ed" containerID="85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e" exitCode=0
Jul 09 16:32:18.306164 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.306134    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv"
Jul 09 16:32:18.306164 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.306156    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv" event={"ID":"16d1b4e7-a004-4f82-b0c1-f129769690ed","Type":"ContainerDied","Data":"85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e"}
Jul 09 16:32:18.306667 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.306197    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5896f8fb9f-7j7lv" event={"ID":"16d1b4e7-a004-4f82-b0c1-f129769690ed","Type":"ContainerDied","Data":"0fbcde281ff6c7e7f6365d52a52bb598ea200e8a689df062f2977242962ab409"}
Jul 09 16:32:18.306667 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.306213    2586 scope.go:117] "RemoveContainer" containerID="85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e"
Jul 09 16:32:18.315303 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.315286    2586 scope.go:117] "RemoveContainer" containerID="85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e"
Jul 09 16:32:18.315539 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:32:18.315521    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e\": container with ID starting with 85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e not found: ID does not exist" containerID="85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e"
Jul 09 16:32:18.315603 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.315546    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e"} err="failed to get container status \"85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e\": rpc error: code = NotFound desc = could not find container \"85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e\": container with ID starting with 85dcec934d84f212ee78c804ced9620c10a90fb9252ba6855d01d560e750e35e not found: ID does not exist"
Jul 09 16:32:18.331872 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.331841    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5896f8fb9f-7j7lv"]
Jul 09 16:32:18.336273 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.336249    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5896f8fb9f-7j7lv"]
Jul 09 16:32:18.772549 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:18.772518    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d1b4e7-a004-4f82-b0c1-f129769690ed" path="/var/lib/kubelet/pods/16d1b4e7-a004-4f82-b0c1-f129769690ed/volumes"
Jul 09 16:32:44.637193 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:44.637152    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:32:54.126754 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:32:54.126709    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:33:05.228740 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:33:05.228697    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:33:14.727409 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:33:14.727330    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:33:25.935815 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:33:25.935778    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:33:53.438811 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:33:53.438769    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:34:05.935168 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:34:05.935131    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:34:40.432987 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:34:40.432945    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:34:50.133858 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:34:50.133772    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:34:50.685189 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:34:50.685162    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:34:50.687752 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:34:50.687729    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:34:58.728185 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:34:58.728151    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:35:06.447459 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:35:06.447409    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:35:17.628182 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:35:17.628147    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:35:26.932993 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:35:26.932956    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:35:37.927725 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:35:37.927689    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:35:47.523444 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:35:47.523410    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:36:10.225245 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:36:10.225207    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:36:21.725942 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:36:21.725866    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:36:44.452390 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:36:44.452351    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:36:54.930132 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:36:54.930094    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:37:10.936408 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:37:10.936369    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:37:19.134214 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:37:19.134175    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:37:32.523492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:37:32.523448    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:37:41.126473 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:37:41.126389    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:37:50.531583 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:37:50.531542    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:39:39.250582 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:39:39.250507    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:39:48.633877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:39:48.633833    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:39:50.722139 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:39:50.722107    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:39:50.724215 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:39:50.724197    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:40:00.847968 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:40:00.847931    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:40:12.144207 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:40:12.144170    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:40:24.134881 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:40:24.134835    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:40:33.537285 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:40:33.537249    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:40:43.954214 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:40:43.954126    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:40:53.539614 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:40:53.539581    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:41:05.832194 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:41:05.832157    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:41:14.730307 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:41:14.730272    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:41:25.135635 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:41:25.135596    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:41:49.341995 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:41:49.341913    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:41:57.237624 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:41:57.237587    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:42:07.137994 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:42:07.137957    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:42:16.936400 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:42:16.936306    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:42:26.538138 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:42:26.538106    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:42:39.239826 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:42:39.239788    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:42:53.639275 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:42:53.639237    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:43:02.533206 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:43:02.533167    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:43:15.871987 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:43:15.871919    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:43:26.341119 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:43:26.341084    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:43:39.431252 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:43:39.431211    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:43:49.037404 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:43:49.037315    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:43:58.543853 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:43:58.543817    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:44:08.037018 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:08.036983    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:44:21.036407 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:21.036362    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:44:29.936060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:29.936022    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:44:42.935148 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:42.935110    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:44:50.756659 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:50.756629    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:44:50.759777 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:50.759742    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:44:54.947233 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:44:54.947201    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:45:05.830329 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:45:05.830280    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:45:13.028530 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:45:13.028493    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:45:26.132028 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:45:26.131990    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:45:35.226713 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:45:35.226677    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:45:44.385873 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:45:44.385834    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:46:05.143618 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:46:05.143583    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:46:20.233895 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:46:20.233857    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:46:35.143628 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:46:35.143586    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:46:45.833155 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:46:45.833067    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:48:30.026677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:48:30.026595    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:48:39.228260 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:48:39.228222    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:49:50.791501 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:49:50.791476    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:49:50.795188 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:49:50.795168    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:49:56.734411 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:49:56.734375    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:50:08.132613 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:08.132578    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:50:17.723835 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:17.723796    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:50:42.624212 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:42.624174    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:50:53.251361 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.251324    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"]
Jul 09 16:50:53.251966 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.251694    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d1b4e7-a004-4f82-b0c1-f129769690ed" containerName="authorino"
Jul 09 16:50:53.251966 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.251705    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d1b4e7-a004-4f82-b0c1-f129769690ed" containerName="authorino"
Jul 09 16:50:53.251966 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.251788    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d1b4e7-a004-4f82-b0c1-f129769690ed" containerName="authorino"
Jul 09 16:50:53.254793 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.254752    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.258822 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.258790    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-756fbf8e\"/\"e2e-lifecycle-model-756fbf8e-kserve-self-signed-certs\""
Jul 09 16:50:53.258822 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.258802    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-756fbf8e\"/\"default-dockercfg-52mm9\""
Jul 09 16:50:53.258822 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.258791    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-756fbf8e\"/\"kube-root-ca.crt\""
Jul 09 16:50:53.259038 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.258945    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-756fbf8e\"/\"openshift-service-ca.crt\""
Jul 09 16:50:53.266464 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.266440    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"]
Jul 09 16:50:53.329662 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.329624    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tmp-dir\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.329849 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.329672    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-model-cache\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.329849 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.329711    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vdr2\" (UniqueName: \"kubernetes.io/projected/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-kube-api-access-6vdr2\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.329849 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.329747    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-dshm\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.329849 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.329796    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tls-certs\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.329849 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.329826    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-home\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.430858 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.430803    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tls-certs\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.430876    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-home\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.430923    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tmp-dir\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.430963    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-model-cache\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.430999    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vdr2\" (UniqueName: \"kubernetes.io/projected/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-kube-api-access-6vdr2\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431069 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.431047    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-dshm\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431349 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.431325    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-home\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431439 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.431378    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tmp-dir\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.431439 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.431389    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-model-cache\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.433210 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.433186    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-dshm\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.433502 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.433481    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tls-certs\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.439867 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.439844    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vdr2\" (UniqueName: \"kubernetes.io/projected/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-kube-api-access-6vdr2\") pod \"e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") " pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.564702 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.564615    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:50:53.697592 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:53.697564    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"]
Jul 09 16:50:53.699462 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:50:53.699425    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a896aa3_01a7_4b9f_8d86_58ddc15c516e.slice/crio-b273f95be0903e541b783d4b5460f91868db05860aa1add46427e67e8260c4dc WatchSource:0}: Error finding container b273f95be0903e541b783d4b5460f91868db05860aa1add46427e67e8260c4dc: Status 404 returned error can't find the container with id b273f95be0903e541b783d4b5460f91868db05860aa1add46427e67e8260c4dc
Jul 09 16:50:54.046961 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:54.046920    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx" event={"ID":"6a896aa3-01a7-4b9f-8d86-58ddc15c516e","Type":"ContainerStarted","Data":"5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f"}
Jul 09 16:50:54.046961 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:54.046963    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx" event={"ID":"6a896aa3-01a7-4b9f-8d86-58ddc15c516e","Type":"ContainerStarted","Data":"b273f95be0903e541b783d4b5460f91868db05860aa1add46427e67e8260c4dc"}
Jul 09 16:50:54.063127 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:54.063078    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx" podStartSLOduration=1.063061074 podStartE2EDuration="1.063061074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:50:54.061587475 +0000 UTC m=+1864.987541753" watchObservedRunningTime="2026-07-09 16:50:54.063061074 +0000 UTC m=+1864.989015446"
Jul 09 16:50:57.321101 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:50:57.321069    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:51:03.564915 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:03.564870    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:51:03.565397 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:03.564936    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:51:03.577704 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:03.577673    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:51:04.109028 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:04.108995    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:51:34.121828 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:34.121738    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:51:44.523095 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:44.523063    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:51:50.407193 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:50.407152    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"]
Jul 09 16:51:54.027355 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:51:54.027322    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:52:05.427708 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.427636    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx" podUID="6a896aa3-01a7-4b9f-8d86-58ddc15c516e" containerName="main" containerID="cri-o://5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f" gracePeriod=45
Jul 09 16:52:05.677577 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.677545    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:52:05.767060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.766969    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-dshm\") pod \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") "
Jul 09 16:52:05.767060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767013    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tls-certs\") pod \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") "
Jul 09 16:52:05.767060 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767057    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-model-cache\") pod \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") "
Jul 09 16:52:05.767337 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767083    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tmp-dir\") pod \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") "
Jul 09 16:52:05.767337 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767137    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vdr2\" (UniqueName: \"kubernetes.io/projected/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-kube-api-access-6vdr2\") pod \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") "
Jul 09 16:52:05.767337 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767210    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-home\") pod \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\" (UID: \"6a896aa3-01a7-4b9f-8d86-58ddc15c516e\") "
Jul 09 16:52:05.767481 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767380    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a896aa3-01a7-4b9f-8d86-58ddc15c516e" (UID: "6a896aa3-01a7-4b9f-8d86-58ddc15c516e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:52:05.767481 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767391    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a896aa3-01a7-4b9f-8d86-58ddc15c516e" (UID: "6a896aa3-01a7-4b9f-8d86-58ddc15c516e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:52:05.767596 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767494    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:52:05.767596 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767495    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-home" (OuterVolumeSpecName: "home") pod "6a896aa3-01a7-4b9f-8d86-58ddc15c516e" (UID: "6a896aa3-01a7-4b9f-8d86-58ddc15c516e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:52:05.767596 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.767511    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:52:05.769357 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.769331    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-kube-api-access-6vdr2" (OuterVolumeSpecName: "kube-api-access-6vdr2") pod "6a896aa3-01a7-4b9f-8d86-58ddc15c516e" (UID: "6a896aa3-01a7-4b9f-8d86-58ddc15c516e"). InnerVolumeSpecName "kube-api-access-6vdr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:52:05.769357 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.769344    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a896aa3-01a7-4b9f-8d86-58ddc15c516e" (UID: "6a896aa3-01a7-4b9f-8d86-58ddc15c516e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:52:05.769690 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.769414    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-dshm" (OuterVolumeSpecName: "dshm") pod "6a896aa3-01a7-4b9f-8d86-58ddc15c516e" (UID: "6a896aa3-01a7-4b9f-8d86-58ddc15c516e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:52:05.868616 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.868575    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:52:05.868616 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.868608    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:52:05.868616 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.868620    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vdr2\" (UniqueName: \"kubernetes.io/projected/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-kube-api-access-6vdr2\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:52:05.868616 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:05.868629    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a896aa3-01a7-4b9f-8d86-58ddc15c516e-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:52:06.423000 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.422967    2586 generic.go:356] "Generic (PLEG): container finished" podID="6a896aa3-01a7-4b9f-8d86-58ddc15c516e" containerID="5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f" exitCode=0
Jul 09 16:52:06.423149 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.423006    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"
Jul 09 16:52:06.423149 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.423044    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx" event={"ID":"6a896aa3-01a7-4b9f-8d86-58ddc15c516e","Type":"ContainerDied","Data":"5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f"}
Jul 09 16:52:06.423149 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.423084    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx" event={"ID":"6a896aa3-01a7-4b9f-8d86-58ddc15c516e","Type":"ContainerDied","Data":"b273f95be0903e541b783d4b5460f91868db05860aa1add46427e67e8260c4dc"}
Jul 09 16:52:06.423149 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.423105    2586 scope.go:117] "RemoveContainer" containerID="5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f"
Jul 09 16:52:06.433024 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.432825    2586 scope.go:117] "RemoveContainer" containerID="5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f"
Jul 09 16:52:06.433275 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:52:06.433136    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f\": container with ID starting with 5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f not found: ID does not exist" containerID="5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f"
Jul 09 16:52:06.433275 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.433161    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f"} err="failed to get container status \"5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f\": rpc error: code = NotFound desc = could not find container \"5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f\": container with ID starting with 5492501a435717ffc1ec889103ced40584972fe949a0031f28db50d3709dba2f not found: ID does not exist"
Jul 09 16:52:06.444696 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.444670    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"]
Jul 09 16:52:06.446657 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.446638    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-756fbf8e/e2e-lifecycle-model-756fbf8e-kserve-fb44846d7-8ljjx"]
Jul 09 16:52:06.772270 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:06.772196    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a896aa3-01a7-4b9f-8d86-58ddc15c516e" path="/var/lib/kubelet/pods/6a896aa3-01a7-4b9f-8d86-58ddc15c516e/volumes"
Jul 09 16:52:30.528190 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:30.528156    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:52:43.828005 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:43.827912    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:52:54.023833 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:52:54.023791    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:54:10.443135 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:54:10.443094    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:54:50.826938 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:54:50.826909    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:54:50.830306 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:54:50.830286    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:54:58.037644 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:54:58.037606    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:55:12.627119 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:12.627082    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:55:20.225671 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.225636    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"]
Jul 09 16:55:20.228218 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.226021    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a896aa3-01a7-4b9f-8d86-58ddc15c516e" containerName="main"
Jul 09 16:55:20.228218 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.226033    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a896aa3-01a7-4b9f-8d86-58ddc15c516e" containerName="main"
Jul 09 16:55:20.228218 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.226108    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a896aa3-01a7-4b9f-8d86-58ddc15c516e" containerName="main"
Jul 09 16:55:20.229103 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.229087    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.232573 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.232548    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9d86cf\"/\"kube-root-ca.crt\""
Jul 09 16:55:20.232709 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.232691    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9d86cf\"/\"openshift-service-ca.crt\""
Jul 09 16:55:20.232786 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.232689    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9d86cf\"/\"test-model-9d86cf-kserve-self-signed-certs\""
Jul 09 16:55:20.232869 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.232853    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9d86cf\"/\"default-dockercfg-nc9p5\""
Jul 09 16:55:20.239497 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.239475    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"]
Jul 09 16:55:20.340601 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.340562    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-home\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.340794 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.340605    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssjt6\" (UniqueName: \"kubernetes.io/projected/f0dbe652-1096-4294-a4a5-d091386d4ff1-kube-api-access-ssjt6\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.340794 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.340644    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-model-cache\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.340794 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.340698    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-dshm\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.340794 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.340715    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-tmp-dir\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.340794 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.340752    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0dbe652-1096-4294-a4a5-d091386d4ff1-tls-certs\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.441953 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.441917    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-dshm\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.441953 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.441952    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-tmp-dir\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442205 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.441981    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0dbe652-1096-4294-a4a5-d091386d4ff1-tls-certs\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442205 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.442050    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-home\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442205 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.442080    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssjt6\" (UniqueName: \"kubernetes.io/projected/f0dbe652-1096-4294-a4a5-d091386d4ff1-kube-api-access-ssjt6\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442205 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.442133    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-model-cache\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442430 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.442390    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-tmp-dir\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442486 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.442434    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-home\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.442528 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.442501    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-model-cache\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.444322 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.444298    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-dshm\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.444592 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.444574    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0dbe652-1096-4294-a4a5-d091386d4ff1-tls-certs\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.450248 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.450228    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssjt6\" (UniqueName: \"kubernetes.io/projected/f0dbe652-1096-4294-a4a5-d091386d4ff1-kube-api-access-ssjt6\") pod \"test-model-9d86cf-kserve-64d44d45d5-c9dmg\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") " pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.538597 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.538504    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:20.671389 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:20.671362    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"]
Jul 09 16:55:20.672830 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:55:20.672802    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0dbe652_1096_4294_a4a5_d091386d4ff1.slice/crio-5e20b37bd5fecd988b81e77713706476394ba3a5ec7d2c9896ce9b12e5499310 WatchSource:0}: Error finding container 5e20b37bd5fecd988b81e77713706476394ba3a5ec7d2c9896ce9b12e5499310: Status 404 returned error can't find the container with id 5e20b37bd5fecd988b81e77713706476394ba3a5ec7d2c9896ce9b12e5499310
Jul 09 16:55:21.423386 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:21.423350    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg" event={"ID":"f0dbe652-1096-4294-a4a5-d091386d4ff1","Type":"ContainerStarted","Data":"242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76"}
Jul 09 16:55:21.423828 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:21.423400    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg" event={"ID":"f0dbe652-1096-4294-a4a5-d091386d4ff1","Type":"ContainerStarted","Data":"5e20b37bd5fecd988b81e77713706476394ba3a5ec7d2c9896ce9b12e5499310"}
Jul 09 16:55:21.441661 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:21.441612    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg" podStartSLOduration=1.441596779 podStartE2EDuration="1.441596779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:55:21.438687042 +0000 UTC m=+2132.364641320" watchObservedRunningTime="2026-07-09 16:55:21.441596779 +0000 UTC m=+2132.367551057"
Jul 09 16:55:25.631244 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:25.631210    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:55:30.538725 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:30.538684    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:30.539314 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:30.538749    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:30.551417 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:30.551383    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:31.486292 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:31.486259    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:55:50.624658 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.624576    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"]
Jul 09 16:55:50.628182 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.628160    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.630814 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.630788    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1876d0\"/\"openshift-service-ca.crt\""
Jul 09 16:55:50.631942 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.631918    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1876d0\"/\"kube-root-ca.crt\""
Jul 09 16:55:50.632048 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.632014    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1876d0\"/\"test-model-1876d0-kserve-self-signed-certs\""
Jul 09 16:55:50.632048 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.632027    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1876d0\"/\"default-dockercfg-sjkxf\""
Jul 09 16:55:50.637610 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.637588    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"]
Jul 09 16:55:50.712398 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.712356    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-dshm\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.712398 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.712395    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z6g9\" (UniqueName: \"kubernetes.io/projected/7c41c2b8-f507-4174-99ee-b44496b85870-kube-api-access-7z6g9\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.712600 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.712469    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-tmp-dir\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.712600 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.712524    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-home\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.712600 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.712543    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c41c2b8-f507-4174-99ee-b44496b85870-tls-certs\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.712600 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.712573    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-model-cache\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.812971 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.812944    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-tmp-dir\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813152 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.812986    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-home\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813152 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813011    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c41c2b8-f507-4174-99ee-b44496b85870-tls-certs\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813152 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813046    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-model-cache\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813152 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813105    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-dshm\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813152 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813132    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z6g9\" (UniqueName: \"kubernetes.io/projected/7c41c2b8-f507-4174-99ee-b44496b85870-kube-api-access-7z6g9\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813459 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813433    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-tmp-dir\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813459 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813437    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-home\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.813597 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.813504    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-model-cache\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.815523 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.815498    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-dshm\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.815697 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.815679    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c41c2b8-f507-4174-99ee-b44496b85870-tls-certs\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.821045 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.821014    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z6g9\" (UniqueName: \"kubernetes.io/projected/7c41c2b8-f507-4174-99ee-b44496b85870-kube-api-access-7z6g9\") pod \"test-model-1876d0-kserve-5c69f45f4f-hhxzs\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") " pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:50.938954 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:50.938871    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:55:51.071830 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:51.071799    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"]
Jul 09 16:55:51.073920 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:55:51.073893    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c41c2b8_f507_4174_99ee_b44496b85870.slice/crio-acb826aac9316076ecac48943b6caa0483f63979e216862f916785fabee2fd1b WatchSource:0}: Error finding container acb826aac9316076ecac48943b6caa0483f63979e216862f916785fabee2fd1b: Status 404 returned error can't find the container with id acb826aac9316076ecac48943b6caa0483f63979e216862f916785fabee2fd1b
Jul 09 16:55:51.582815 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:51.582779    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs" event={"ID":"7c41c2b8-f507-4174-99ee-b44496b85870","Type":"ContainerStarted","Data":"8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40"}
Jul 09 16:55:51.583058 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:51.582822    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs" event={"ID":"7c41c2b8-f507-4174-99ee-b44496b85870","Type":"ContainerStarted","Data":"acb826aac9316076ecac48943b6caa0483f63979e216862f916785fabee2fd1b"}
Jul 09 16:55:51.597749 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:51.597696    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs" podStartSLOduration=1.597681091 podStartE2EDuration="1.597681091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:55:51.596940723 +0000 UTC m=+2162.522895001" watchObservedRunningTime="2026-07-09 16:55:51.597681091 +0000 UTC m=+2162.523635369"
Jul 09 16:55:54.828193 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:55:54.828150    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:56:00.939532 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:00.939483    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:56:00.939532 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:00.939540    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:56:00.952336 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:00.952307    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:56:01.646850 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:01.646817    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:56:08.431472 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:08.431437    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:56:28.830013 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:28.829980    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:56:33.632319 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.632284    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"]
Jul 09 16:56:33.635770 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.635745    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.638162 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.638143    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"kube-root-ca.crt\""
Jul 09 16:56:33.638295 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.638240    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"openshift-service-ca.crt\""
Jul 09 16:56:33.639286 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.639266    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"test-model-7ab20f-kserve-self-signed-certs\""
Jul 09 16:56:33.639393 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.639324    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"default-dockercfg-c9td7\""
Jul 09 16:56:33.643735 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.643715    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"]
Jul 09 16:56:33.802328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.802295    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-model-cache\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.802515 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.802359    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-tmp-dir\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.802515 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.802411    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d7f2c-6252-4de8-aa76-344c469864b7-tls-certs\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.802515 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.802446    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-home\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.802515 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.802469    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-dshm\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.802515 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.802506    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jhb\" (UniqueName: \"kubernetes.io/projected/4c3d7f2c-6252-4de8-aa76-344c469864b7-kube-api-access-42jhb\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903176 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903071    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jhb\" (UniqueName: \"kubernetes.io/projected/4c3d7f2c-6252-4de8-aa76-344c469864b7-kube-api-access-42jhb\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903363 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903297    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-model-cache\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903440 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903389    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-tmp-dir\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903440 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903421    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d7f2c-6252-4de8-aa76-344c469864b7-tls-certs\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903464    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-home\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903531 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903487    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-dshm\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903812 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903787    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-model-cache\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903893 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903795    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-tmp-dir\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.903893 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.903876    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-home\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.905732 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.905710    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-dshm\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.905882 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.905866    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d7f2c-6252-4de8-aa76-344c469864b7-tls-certs\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.910789 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.910741    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jhb\" (UniqueName: \"kubernetes.io/projected/4c3d7f2c-6252-4de8-aa76-344c469864b7-kube-api-access-42jhb\") pod \"test-model-7ab20f-kserve-766975f9b5-mz66j\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:33.946631 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:33.946604    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:34.084484 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:34.084456    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"]
Jul 09 16:56:34.086155 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:56:34.086128    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c3d7f2c_6252_4de8_aa76_344c469864b7.slice/crio-69d83929d7689a62bbbbd2f9befb1edad75805a740134861e78d291067602b27 WatchSource:0}: Error finding container 69d83929d7689a62bbbbd2f9befb1edad75805a740134861e78d291067602b27: Status 404 returned error can't find the container with id 69d83929d7689a62bbbbd2f9befb1edad75805a740134861e78d291067602b27
Jul 09 16:56:34.816127 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:34.816088    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j" event={"ID":"4c3d7f2c-6252-4de8-aa76-344c469864b7","Type":"ContainerStarted","Data":"3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f"}
Jul 09 16:56:34.816127 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:34.816134    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j" event={"ID":"4c3d7f2c-6252-4de8-aa76-344c469864b7","Type":"ContainerStarted","Data":"69d83929d7689a62bbbbd2f9befb1edad75805a740134861e78d291067602b27"}
Jul 09 16:56:34.833894 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:34.833831    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j" podStartSLOduration=1.833813578 podStartE2EDuration="1.833813578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:56:34.831428888 +0000 UTC m=+2205.757383167" watchObservedRunningTime="2026-07-09 16:56:34.833813578 +0000 UTC m=+2205.759767855"
Jul 09 16:56:43.947490 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:43.947451    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:43.948006 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:43.947509    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:43.960090 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:43.960064    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:44.879568 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:44.879542    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:56:52.330919 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:56:52.330883    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:57:02.733694 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:02.733654    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"]
Jul 09 16:57:17.754653 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:17.754540    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j" podUID="4c3d7f2c-6252-4de8-aa76-344c469864b7" containerName="main" containerID="cri-o://3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f" gracePeriod=45
Jul 09 16:57:17.928960 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:17.928924    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:57:18.000482 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.000458    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:57:18.051421 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.051335    2586 generic.go:356] "Generic (PLEG): container finished" podID="4c3d7f2c-6252-4de8-aa76-344c469864b7" containerID="3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f" exitCode=0
Jul 09 16:57:18.051421 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.051380    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"
Jul 09 16:57:18.051617 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.051413    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j" event={"ID":"4c3d7f2c-6252-4de8-aa76-344c469864b7","Type":"ContainerDied","Data":"3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f"}
Jul 09 16:57:18.051617 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.051459    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j" event={"ID":"4c3d7f2c-6252-4de8-aa76-344c469864b7","Type":"ContainerDied","Data":"69d83929d7689a62bbbbd2f9befb1edad75805a740134861e78d291067602b27"}
Jul 09 16:57:18.051617 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.051477    2586 scope.go:117] "RemoveContainer" containerID="3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f"
Jul 09 16:57:18.060498 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.060480    2586 scope.go:117] "RemoveContainer" containerID="3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f"
Jul 09 16:57:18.060855 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:57:18.060819    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f\": container with ID starting with 3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f not found: ID does not exist" containerID="3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f"
Jul 09 16:57:18.060972 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.060861    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f"} err="failed to get container status \"3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f\": rpc error: code = NotFound desc = could not find container \"3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f\": container with ID starting with 3251b39485d2707b7b33d580e4bc998d0f850667f487f49114e1e085bc75e92f not found: ID does not exist"
Jul 09 16:57:18.098449 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098421    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-tmp-dir\") pod \"4c3d7f2c-6252-4de8-aa76-344c469864b7\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") "
Jul 09 16:57:18.098449 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098450    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-dshm\") pod \"4c3d7f2c-6252-4de8-aa76-344c469864b7\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") "
Jul 09 16:57:18.098653 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098470    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42jhb\" (UniqueName: \"kubernetes.io/projected/4c3d7f2c-6252-4de8-aa76-344c469864b7-kube-api-access-42jhb\") pod \"4c3d7f2c-6252-4de8-aa76-344c469864b7\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") "
Jul 09 16:57:18.098653 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098505    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-model-cache\") pod \"4c3d7f2c-6252-4de8-aa76-344c469864b7\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") "
Jul 09 16:57:18.098653 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098558    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-home\") pod \"4c3d7f2c-6252-4de8-aa76-344c469864b7\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") "
Jul 09 16:57:18.098653 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098582    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d7f2c-6252-4de8-aa76-344c469864b7-tls-certs\") pod \"4c3d7f2c-6252-4de8-aa76-344c469864b7\" (UID: \"4c3d7f2c-6252-4de8-aa76-344c469864b7\") "
Jul 09 16:57:18.099300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098784    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c3d7f2c-6252-4de8-aa76-344c469864b7" (UID: "4c3d7f2c-6252-4de8-aa76-344c469864b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:57:18.099300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098878    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c3d7f2c-6252-4de8-aa76-344c469864b7" (UID: "4c3d7f2c-6252-4de8-aa76-344c469864b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:57:18.099300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098886    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-home" (OuterVolumeSpecName: "home") pod "4c3d7f2c-6252-4de8-aa76-344c469864b7" (UID: "4c3d7f2c-6252-4de8-aa76-344c469864b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:57:18.099300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098970    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:57:18.099300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.098989    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:57:18.099300 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.099001    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:57:18.100938 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.100909    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-dshm" (OuterVolumeSpecName: "dshm") pod "4c3d7f2c-6252-4de8-aa76-344c469864b7" (UID: "4c3d7f2c-6252-4de8-aa76-344c469864b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:57:18.100938 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.100919    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3d7f2c-6252-4de8-aa76-344c469864b7-kube-api-access-42jhb" (OuterVolumeSpecName: "kube-api-access-42jhb") pod "4c3d7f2c-6252-4de8-aa76-344c469864b7" (UID: "4c3d7f2c-6252-4de8-aa76-344c469864b7"). InnerVolumeSpecName "kube-api-access-42jhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:57:18.101064 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.100960    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c3d7f2c-6252-4de8-aa76-344c469864b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c3d7f2c-6252-4de8-aa76-344c469864b7" (UID: "4c3d7f2c-6252-4de8-aa76-344c469864b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:57:18.200155 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.200119    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c3d7f2c-6252-4de8-aa76-344c469864b7-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:57:18.200155 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.200151    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c3d7f2c-6252-4de8-aa76-344c469864b7-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:57:18.200155 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.200161    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42jhb\" (UniqueName: \"kubernetes.io/projected/4c3d7f2c-6252-4de8-aa76-344c469864b7-kube-api-access-42jhb\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:57:18.373081 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.373050    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"]
Jul 09 16:57:18.376530 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.376503    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-mz66j"]
Jul 09 16:57:18.773389 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:18.773309    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c3d7f2c-6252-4de8-aa76-344c469864b7" path="/var/lib/kubelet/pods/4c3d7f2c-6252-4de8-aa76-344c469864b7/volumes"
Jul 09 16:57:44.420406 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:44.420372    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:57:48.451199 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.451158    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"]
Jul 09 16:57:48.451683 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.451539    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c3d7f2c-6252-4de8-aa76-344c469864b7" containerName="main"
Jul 09 16:57:48.451683 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.451551    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c3d7f2c-6252-4de8-aa76-344c469864b7" containerName="main"
Jul 09 16:57:48.451683 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.451613    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c3d7f2c-6252-4de8-aa76-344c469864b7" containerName="main"
Jul 09 16:57:48.454718 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.454698    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.456924 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.456905    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 16:57:48.457050 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.456930    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-reuse-7ab20f-serving-cert\""
Jul 09 16:57:48.457795 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.457775    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h2fl4\""
Jul 09 16:57:48.457910 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.457784    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 16:57:48.465677 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.465656    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"]
Jul 09 16:57:48.566934 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.566891    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235ee757-f61a-47d1-add3-c9e780be65de-openshift-service-ca\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.566934 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.566939    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7m7j\" (UniqueName: \"kubernetes.io/projected/235ee757-f61a-47d1-add3-c9e780be65de-kube-api-access-w7m7j\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.567179 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.566973    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.667700 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.667669    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235ee757-f61a-47d1-add3-c9e780be65de-openshift-service-ca\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.667914 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.667709    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7m7j\" (UniqueName: \"kubernetes.io/projected/235ee757-f61a-47d1-add3-c9e780be65de-kube-api-access-w7m7j\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.667914 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.667734    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.667914 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:57:48.667863    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-serving-cert: secret "maas-api-e2e-reuse-7ab20f-serving-cert" not found
Jul 09 16:57:48.668181 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:57:48.667929    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls podName:235ee757-f61a-47d1-add3-c9e780be65de nodeName:}" failed. No retries permitted until 2026-07-09 16:57:49.167913145 +0000 UTC m=+2280.093867401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls") pod "maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" (UID: "235ee757-f61a-47d1-add3-c9e780be65de") : secret "maas-api-e2e-reuse-7ab20f-serving-cert" not found
Jul 09 16:57:48.668306 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.668282    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235ee757-f61a-47d1-add3-c9e780be65de-openshift-service-ca\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:48.682373 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:48.682342    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7m7j\" (UniqueName: \"kubernetes.io/projected/235ee757-f61a-47d1-add3-c9e780be65de-kube-api-access-w7m7j\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:49.171853 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:49.171812    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:49.174420 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:49.174383    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls\") pod \"maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") " pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:49.366413 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:49.366376    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:49.499855 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:49.499797    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"]
Jul 09 16:57:49.504341 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:49.504315    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 16:57:50.222374 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:50.222331    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" event={"ID":"235ee757-f61a-47d1-add3-c9e780be65de","Type":"ContainerStarted","Data":"a88082e3489890415f28d0b8b8e78f0d128a49057e0f4d8a988cf73793ac986b"}
Jul 09 16:57:52.235779 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:52.235726    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" event={"ID":"235ee757-f61a-47d1-add3-c9e780be65de","Type":"ContainerStarted","Data":"b4b164701410d7f2ce6161a09e7e78ecbc9a7f84a75e69e4e6a15012f24ba66a"}
Jul 09 16:57:52.236171 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:52.235873    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:57:52.252705 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:52.252655    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" podStartSLOduration=2.204112881 podStartE2EDuration="4.252641336s" totalImagesPullingTime="2.048528455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:57:52.250354396 +0000 UTC m=+2283.176308654" watchObservedRunningTime="2026-07-09 16:57:52.252641336 +0000 UTC m=+2283.178595637"
Jul 09 16:57:58.247144 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:57:58.247117    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:58:05.436089 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.436053    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"]
Jul 09 16:58:05.439624 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.439598    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.443189 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.443151    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"default-dockercfg-4gwjg\""
Jul 09 16:58:05.443189 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.443178    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"openshift-service-ca.crt\""
Jul 09 16:58:05.443398 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.443193    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"kube-root-ca.crt\""
Jul 09 16:58:05.443398 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.443151    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-reuse-7ab20f\"/\"test-model-7ab20f-kserve-self-signed-certs\""
Jul 09 16:58:05.454661 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.454641    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"]
Jul 09 16:58:05.621399 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.621358    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7rtc\" (UniqueName: \"kubernetes.io/projected/91308378-f46f-435a-b42b-b50d8f0d4f1b-kube-api-access-z7rtc\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.621579 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.621430    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-model-cache\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.621579 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.621465    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-home\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.621579 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.621486    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-tmp-dir\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.621579 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.621505    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-dshm\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.621579 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.621531    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91308378-f46f-435a-b42b-b50d8f0d4f1b-tls-certs\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.722829 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.722713    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-tmp-dir\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.722829 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.722755    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-dshm\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.722829 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.722816    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91308378-f46f-435a-b42b-b50d8f0d4f1b-tls-certs\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.723049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.722877    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7rtc\" (UniqueName: \"kubernetes.io/projected/91308378-f46f-435a-b42b-b50d8f0d4f1b-kube-api-access-z7rtc\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.723049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.722935    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-model-cache\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.723049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.722971    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-home\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.723158 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.723123    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-tmp-dir\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.723303 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.723277    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-home\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.723303 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.723292    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-model-cache\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.725132 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.725110    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-dshm\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.725355 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.725337    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91308378-f46f-435a-b42b-b50d8f0d4f1b-tls-certs\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.730530 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.730505    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7rtc\" (UniqueName: \"kubernetes.io/projected/91308378-f46f-435a-b42b-b50d8f0d4f1b-kube-api-access-z7rtc\") pod \"test-model-7ab20f-kserve-766975f9b5-ln7ml\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") " pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.748448 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.748422    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:05.879150 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:05.879129    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"]
Jul 09 16:58:05.881119 ip-10-0-139-176 kubenswrapper[2586]: W0709 16:58:05.881086    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91308378_f46f_435a_b42b_b50d8f0d4f1b.slice/crio-e9ca2ca6be6569cab5352030557eab54d492f04af5721e566fe74ed8bd9d233b WatchSource:0}: Error finding container e9ca2ca6be6569cab5352030557eab54d492f04af5721e566fe74ed8bd9d233b: Status 404 returned error can't find the container with id e9ca2ca6be6569cab5352030557eab54d492f04af5721e566fe74ed8bd9d233b
Jul 09 16:58:06.310388 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:06.310350    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml" event={"ID":"91308378-f46f-435a-b42b-b50d8f0d4f1b","Type":"ContainerStarted","Data":"a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae"}
Jul 09 16:58:06.310612 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:06.310397    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml" event={"ID":"91308378-f46f-435a-b42b-b50d8f0d4f1b","Type":"ContainerStarted","Data":"e9ca2ca6be6569cab5352030557eab54d492f04af5721e566fe74ed8bd9d233b"}
Jul 09 16:58:06.327046 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:06.326986    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml" podStartSLOduration=1.326969571 podStartE2EDuration="1.326969571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 16:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 16:58:06.325600015 +0000 UTC m=+2297.251554290" watchObservedRunningTime="2026-07-09 16:58:06.326969571 +0000 UTC m=+2297.252923849"
Jul 09 16:58:06.331492 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:06.331460    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:58:15.749430 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:15.749387    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:15.749955 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:15.749484    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:15.762821 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:15.762792    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:16.373553 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:16.373523    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:23.731884 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:23.731848    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:58:29.191590 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.191553    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"]
Jul 09 16:58:29.192074 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.191844    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" podUID="235ee757-f61a-47d1-add3-c9e780be65de" containerName="maas-api" containerID="cri-o://b4b164701410d7f2ce6161a09e7e78ecbc9a7f84a75e69e4e6a15012f24ba66a" gracePeriod=30
Jul 09 16:58:29.434351 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.434308    2586 generic.go:356] "Generic (PLEG): container finished" podID="235ee757-f61a-47d1-add3-c9e780be65de" containerID="b4b164701410d7f2ce6161a09e7e78ecbc9a7f84a75e69e4e6a15012f24ba66a" exitCode=0
Jul 09 16:58:29.434523 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.434391    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" event={"ID":"235ee757-f61a-47d1-add3-c9e780be65de","Type":"ContainerDied","Data":"b4b164701410d7f2ce6161a09e7e78ecbc9a7f84a75e69e4e6a15012f24ba66a"}
Jul 09 16:58:29.542708 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.542681    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:58:29.637183 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.637142    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235ee757-f61a-47d1-add3-c9e780be65de-openshift-service-ca\") pod \"235ee757-f61a-47d1-add3-c9e780be65de\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") "
Jul 09 16:58:29.637384 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.637225    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls\") pod \"235ee757-f61a-47d1-add3-c9e780be65de\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") "
Jul 09 16:58:29.637384 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.637285    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7m7j\" (UniqueName: \"kubernetes.io/projected/235ee757-f61a-47d1-add3-c9e780be65de-kube-api-access-w7m7j\") pod \"235ee757-f61a-47d1-add3-c9e780be65de\" (UID: \"235ee757-f61a-47d1-add3-c9e780be65de\") "
Jul 09 16:58:29.637625 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.637580    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/235ee757-f61a-47d1-add3-c9e780be65de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "235ee757-f61a-47d1-add3-c9e780be65de" (UID: "235ee757-f61a-47d1-add3-c9e780be65de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 16:58:29.639484 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.639460    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "235ee757-f61a-47d1-add3-c9e780be65de" (UID: "235ee757-f61a-47d1-add3-c9e780be65de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:58:29.639578 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.639484    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235ee757-f61a-47d1-add3-c9e780be65de-kube-api-access-w7m7j" (OuterVolumeSpecName: "kube-api-access-w7m7j") pod "235ee757-f61a-47d1-add3-c9e780be65de" (UID: "235ee757-f61a-47d1-add3-c9e780be65de"). InnerVolumeSpecName "kube-api-access-w7m7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:58:29.738547 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.738462    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235ee757-f61a-47d1-add3-c9e780be65de-maas-api-tls\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:29.738547 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.738491    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7m7j\" (UniqueName: \"kubernetes.io/projected/235ee757-f61a-47d1-add3-c9e780be65de-kube-api-access-w7m7j\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:29.738547 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:29.738501    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235ee757-f61a-47d1-add3-c9e780be65de-openshift-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:30.441772 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:30.441731    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"
Jul 09 16:58:30.442198 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:30.441781    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth" event={"ID":"235ee757-f61a-47d1-add3-c9e780be65de","Type":"ContainerDied","Data":"a88082e3489890415f28d0b8b8e78f0d128a49057e0f4d8a988cf73793ac986b"}
Jul 09 16:58:30.442198 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:30.441841    2586 scope.go:117] "RemoveContainer" containerID="b4b164701410d7f2ce6161a09e7e78ecbc9a7f84a75e69e4e6a15012f24ba66a"
Jul 09 16:58:30.463903 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:30.463870    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"]
Jul 09 16:58:30.467719 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:30.467697    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-reuse-7ab20f-59b4874d9f-kmcth"]
Jul 09 16:58:30.772656 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:30.772576    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="235ee757-f61a-47d1-add3-c9e780be65de" path="/var/lib/kubelet/pods/235ee757-f61a-47d1-add3-c9e780be65de/volumes"
Jul 09 16:58:34.189136 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:34.189097    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"]
Jul 09 16:58:48.021254 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:48.021158    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:58:49.210044 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.210007    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml" podUID="91308378-f46f-435a-b42b-b50d8f0d4f1b" containerName="main" containerID="cri-o://a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae" gracePeriod=45
Jul 09 16:58:49.459660 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.459634    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:49.542561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.542470    2586 generic.go:356] "Generic (PLEG): container finished" podID="91308378-f46f-435a-b42b-b50d8f0d4f1b" containerID="a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae" exitCode=0
Jul 09 16:58:49.542561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.542513    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"
Jul 09 16:58:49.542561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.542549    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml" event={"ID":"91308378-f46f-435a-b42b-b50d8f0d4f1b","Type":"ContainerDied","Data":"a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae"}
Jul 09 16:58:49.542833 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.542587    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml" event={"ID":"91308378-f46f-435a-b42b-b50d8f0d4f1b","Type":"ContainerDied","Data":"e9ca2ca6be6569cab5352030557eab54d492f04af5721e566fe74ed8bd9d233b"}
Jul 09 16:58:49.542833 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.542605    2586 scope.go:117] "RemoveContainer" containerID="a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae"
Jul 09 16:58:49.551256 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.551239    2586 scope.go:117] "RemoveContainer" containerID="a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae"
Jul 09 16:58:49.551550 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:58:49.551532    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae\": container with ID starting with a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae not found: ID does not exist" containerID="a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae"
Jul 09 16:58:49.551589 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.551561    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae"} err="failed to get container status \"a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae\": rpc error: code = NotFound desc = could not find container \"a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae\": container with ID starting with a3b02c6924399a99800cc6122271e2a908ed40bc75d9c27b19bf1e0b05e047ae not found: ID does not exist"
Jul 09 16:58:49.618121 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618065    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-home\") pod \"91308378-f46f-435a-b42b-b50d8f0d4f1b\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") "
Jul 09 16:58:49.618332 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618157    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-dshm\") pod \"91308378-f46f-435a-b42b-b50d8f0d4f1b\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") "
Jul 09 16:58:49.618332 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618197    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-model-cache\") pod \"91308378-f46f-435a-b42b-b50d8f0d4f1b\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") "
Jul 09 16:58:49.618332 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618225    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-tmp-dir\") pod \"91308378-f46f-435a-b42b-b50d8f0d4f1b\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") "
Jul 09 16:58:49.618332 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618304    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7rtc\" (UniqueName: \"kubernetes.io/projected/91308378-f46f-435a-b42b-b50d8f0d4f1b-kube-api-access-z7rtc\") pod \"91308378-f46f-435a-b42b-b50d8f0d4f1b\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") "
Jul 09 16:58:49.618561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618345    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91308378-f46f-435a-b42b-b50d8f0d4f1b-tls-certs\") pod \"91308378-f46f-435a-b42b-b50d8f0d4f1b\" (UID: \"91308378-f46f-435a-b42b-b50d8f0d4f1b\") "
Jul 09 16:58:49.618561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618363    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-home" (OuterVolumeSpecName: "home") pod "91308378-f46f-435a-b42b-b50d8f0d4f1b" (UID: "91308378-f46f-435a-b42b-b50d8f0d4f1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:58:49.618561 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618511    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "91308378-f46f-435a-b42b-b50d8f0d4f1b" (UID: "91308378-f46f-435a-b42b-b50d8f0d4f1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:58:49.618691 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618568    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "91308378-f46f-435a-b42b-b50d8f0d4f1b" (UID: "91308378-f46f-435a-b42b-b50d8f0d4f1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:58:49.618753 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618728    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:49.618844 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618751    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:49.618844 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.618797    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:49.620520 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.620498    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-dshm" (OuterVolumeSpecName: "dshm") pod "91308378-f46f-435a-b42b-b50d8f0d4f1b" (UID: "91308378-f46f-435a-b42b-b50d8f0d4f1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:58:49.620520 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.620505    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91308378-f46f-435a-b42b-b50d8f0d4f1b-kube-api-access-z7rtc" (OuterVolumeSpecName: "kube-api-access-z7rtc") pod "91308378-f46f-435a-b42b-b50d8f0d4f1b" (UID: "91308378-f46f-435a-b42b-b50d8f0d4f1b"). InnerVolumeSpecName "kube-api-access-z7rtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:58:49.620633 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.620553    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91308378-f46f-435a-b42b-b50d8f0d4f1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "91308378-f46f-435a-b42b-b50d8f0d4f1b" (UID: "91308378-f46f-435a-b42b-b50d8f0d4f1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:58:49.719827 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.719790    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7rtc\" (UniqueName: \"kubernetes.io/projected/91308378-f46f-435a-b42b-b50d8f0d4f1b-kube-api-access-z7rtc\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:49.719827 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.719820    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91308378-f46f-435a-b42b-b50d8f0d4f1b-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:49.719827 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.719829    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91308378-f46f-435a-b42b-b50d8f0d4f1b-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:58:49.863058 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.863028    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"]
Jul 09 16:58:49.866347 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:49.866312    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-reuse-7ab20f/test-model-7ab20f-kserve-766975f9b5-ln7ml"]
Jul 09 16:58:50.773604 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:58:50.773573    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91308378-f46f-435a-b42b-b50d8f0d4f1b" path="/var/lib/kubelet/pods/91308378-f46f-435a-b42b-b50d8f0d4f1b/volumes"
Jul 09 16:59:15.124161 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:15.124114    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:59:18.216485 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:18.216453    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"]
Jul 09 16:59:19.098246 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:19.098211    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"]
Jul 09 16:59:31.926489 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:31.926454    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 16:59:33.715867 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:33.715795    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg" podUID="f0dbe652-1096-4294-a4a5-d091386d4ff1" containerName="main" containerID="cri-o://242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76" gracePeriod=45
Jul 09 16:59:33.964264 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:33.964237    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:59:34.118877 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.118809    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs" podUID="7c41c2b8-f507-4174-99ee-b44496b85870" containerName="main" containerID="cri-o://8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40" gracePeriod=45
Jul 09 16:59:34.120459 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120437    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-home\") pod \"f0dbe652-1096-4294-a4a5-d091386d4ff1\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") "
Jul 09 16:59:34.120566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120509    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-dshm\") pod \"f0dbe652-1096-4294-a4a5-d091386d4ff1\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") "
Jul 09 16:59:34.120566 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120539    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-model-cache\") pod \"f0dbe652-1096-4294-a4a5-d091386d4ff1\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") "
Jul 09 16:59:34.120681 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120584    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0dbe652-1096-4294-a4a5-d091386d4ff1-tls-certs\") pod \"f0dbe652-1096-4294-a4a5-d091386d4ff1\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") "
Jul 09 16:59:34.120681 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120614    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssjt6\" (UniqueName: \"kubernetes.io/projected/f0dbe652-1096-4294-a4a5-d091386d4ff1-kube-api-access-ssjt6\") pod \"f0dbe652-1096-4294-a4a5-d091386d4ff1\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") "
Jul 09 16:59:34.120801 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120687    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-tmp-dir\") pod \"f0dbe652-1096-4294-a4a5-d091386d4ff1\" (UID: \"f0dbe652-1096-4294-a4a5-d091386d4ff1\") "
Jul 09 16:59:34.120801 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120744    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-home" (OuterVolumeSpecName: "home") pod "f0dbe652-1096-4294-a4a5-d091386d4ff1" (UID: "f0dbe652-1096-4294-a4a5-d091386d4ff1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.120911 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120870    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0dbe652-1096-4294-a4a5-d091386d4ff1" (UID: "f0dbe652-1096-4294-a4a5-d091386d4ff1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.121002 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.120983    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.121068 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.121007    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.121130 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.121085    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0dbe652-1096-4294-a4a5-d091386d4ff1" (UID: "f0dbe652-1096-4294-a4a5-d091386d4ff1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.122834 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.122810    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-dshm" (OuterVolumeSpecName: "dshm") pod "f0dbe652-1096-4294-a4a5-d091386d4ff1" (UID: "f0dbe652-1096-4294-a4a5-d091386d4ff1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.122964 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.122913    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0dbe652-1096-4294-a4a5-d091386d4ff1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0dbe652-1096-4294-a4a5-d091386d4ff1" (UID: "f0dbe652-1096-4294-a4a5-d091386d4ff1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:59:34.122964 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.122933    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0dbe652-1096-4294-a4a5-d091386d4ff1-kube-api-access-ssjt6" (OuterVolumeSpecName: "kube-api-access-ssjt6") pod "f0dbe652-1096-4294-a4a5-d091386d4ff1" (UID: "f0dbe652-1096-4294-a4a5-d091386d4ff1"). InnerVolumeSpecName "kube-api-access-ssjt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:59:34.221658 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.221618    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.221658 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.221650    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0dbe652-1096-4294-a4a5-d091386d4ff1-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.221658 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.221663    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssjt6\" (UniqueName: \"kubernetes.io/projected/f0dbe652-1096-4294-a4a5-d091386d4ff1-kube-api-access-ssjt6\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.221930 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.221677    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0dbe652-1096-4294-a4a5-d091386d4ff1-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.354049 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.354025    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:59:34.423074 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.422989    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c41c2b8-f507-4174-99ee-b44496b85870-tls-certs\") pod \"7c41c2b8-f507-4174-99ee-b44496b85870\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") "
Jul 09 16:59:34.423074 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423032    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-home\") pod \"7c41c2b8-f507-4174-99ee-b44496b85870\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") "
Jul 09 16:59:34.423074 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423069    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-model-cache\") pod \"7c41c2b8-f507-4174-99ee-b44496b85870\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") "
Jul 09 16:59:34.423328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423210    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-dshm\") pod \"7c41c2b8-f507-4174-99ee-b44496b85870\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") "
Jul 09 16:59:34.423328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423297    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-tmp-dir\") pod \"7c41c2b8-f507-4174-99ee-b44496b85870\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") "
Jul 09 16:59:34.423328 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423322    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c41c2b8-f507-4174-99ee-b44496b85870" (UID: "7c41c2b8-f507-4174-99ee-b44496b85870"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.423504 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423350    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7z6g9\" (UniqueName: \"kubernetes.io/projected/7c41c2b8-f507-4174-99ee-b44496b85870-kube-api-access-7z6g9\") pod \"7c41c2b8-f507-4174-99ee-b44496b85870\" (UID: \"7c41c2b8-f507-4174-99ee-b44496b85870\") "
Jul 09 16:59:34.423504 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423361    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-home" (OuterVolumeSpecName: "home") pod "7c41c2b8-f507-4174-99ee-b44496b85870" (UID: "7c41c2b8-f507-4174-99ee-b44496b85870"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.423604 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423523    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c41c2b8-f507-4174-99ee-b44496b85870" (UID: "7c41c2b8-f507-4174-99ee-b44496b85870"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.423604 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423567    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.423604 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.423581    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.425346 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.425315    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c41c2b8-f507-4174-99ee-b44496b85870-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c41c2b8-f507-4174-99ee-b44496b85870" (UID: "7c41c2b8-f507-4174-99ee-b44496b85870"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 16:59:34.425426 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.425348    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-dshm" (OuterVolumeSpecName: "dshm") pod "7c41c2b8-f507-4174-99ee-b44496b85870" (UID: "7c41c2b8-f507-4174-99ee-b44496b85870"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 16:59:34.425426 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.425369    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c41c2b8-f507-4174-99ee-b44496b85870-kube-api-access-7z6g9" (OuterVolumeSpecName: "kube-api-access-7z6g9") pod "7c41c2b8-f507-4174-99ee-b44496b85870" (UID: "7c41c2b8-f507-4174-99ee-b44496b85870"). InnerVolumeSpecName "kube-api-access-7z6g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 16:59:34.524025 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.523989    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.524025 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.524019    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c41c2b8-f507-4174-99ee-b44496b85870-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.524231 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.524033    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z6g9\" (UniqueName: \"kubernetes.io/projected/7c41c2b8-f507-4174-99ee-b44496b85870-kube-api-access-7z6g9\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.524231 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.524055    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c41c2b8-f507-4174-99ee-b44496b85870-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 16:59:34.783802 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.783751    2586 generic.go:356] "Generic (PLEG): container finished" podID="7c41c2b8-f507-4174-99ee-b44496b85870" containerID="8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40" exitCode=0
Jul 09 16:59:34.784165 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.783799    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs" event={"ID":"7c41c2b8-f507-4174-99ee-b44496b85870","Type":"ContainerDied","Data":"8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40"}
Jul 09 16:59:34.784165 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.783815    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"
Jul 09 16:59:34.784165 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.783845    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs" event={"ID":"7c41c2b8-f507-4174-99ee-b44496b85870","Type":"ContainerDied","Data":"acb826aac9316076ecac48943b6caa0483f63979e216862f916785fabee2fd1b"}
Jul 09 16:59:34.784165 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.783871    2586 scope.go:117] "RemoveContainer" containerID="8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40"
Jul 09 16:59:34.785292 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.785271    2586 generic.go:356] "Generic (PLEG): container finished" podID="f0dbe652-1096-4294-a4a5-d091386d4ff1" containerID="242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76" exitCode=0
Jul 09 16:59:34.785382 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.785338    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"
Jul 09 16:59:34.785446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.785390    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg" event={"ID":"f0dbe652-1096-4294-a4a5-d091386d4ff1","Type":"ContainerDied","Data":"242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76"}
Jul 09 16:59:34.785446 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.785429    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg" event={"ID":"f0dbe652-1096-4294-a4a5-d091386d4ff1","Type":"ContainerDied","Data":"5e20b37bd5fecd988b81e77713706476394ba3a5ec7d2c9896ce9b12e5499310"}
Jul 09 16:59:34.792900 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.792882    2586 scope.go:117] "RemoveContainer" containerID="8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40"
Jul 09 16:59:34.793141 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:59:34.793122    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40\": container with ID starting with 8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40 not found: ID does not exist" containerID="8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40"
Jul 09 16:59:34.793213 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.793152    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40"} err="failed to get container status \"8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40\": rpc error: code = NotFound desc = could not find container \"8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40\": container with ID starting with 8fc84974a069249e8d3597b5d4eff0b9c214d39fd822298c2d98a7d84be24c40 not found: ID does not exist"
Jul 09 16:59:34.793213 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.793176    2586 scope.go:117] "RemoveContainer" containerID="242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76"
Jul 09 16:59:34.801287 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.801226    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"]
Jul 09 16:59:34.802773 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.802746    2586 scope.go:117] "RemoveContainer" containerID="242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76"
Jul 09 16:59:34.803051 ip-10-0-139-176 kubenswrapper[2586]: E0709 16:59:34.803030    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76\": container with ID starting with 242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76 not found: ID does not exist" containerID="242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76"
Jul 09 16:59:34.803118 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.803060    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76"} err="failed to get container status \"242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76\": rpc error: code = NotFound desc = could not find container \"242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76\": container with ID starting with 242d0878ce491d919ac6a94cfd246fe23847addb1590c9f1a00199182ff8fa76 not found: ID does not exist"
Jul 09 16:59:34.803673 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.803655    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9d86cf/test-model-9d86cf-kserve-64d44d45d5-c9dmg"]
Jul 09 16:59:34.814058 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.814037    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"]
Jul 09 16:59:34.822272 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:34.822249    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1876d0/test-model-1876d0-kserve-5c69f45f4f-hhxzs"]
Jul 09 16:59:36.772508 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:36.772474    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c41c2b8-f507-4174-99ee-b44496b85870" path="/var/lib/kubelet/pods/7c41c2b8-f507-4174-99ee-b44496b85870/volumes"
Jul 09 16:59:36.772883 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:36.772857    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0dbe652-1096-4294-a4a5-d091386d4ff1" path="/var/lib/kubelet/pods/f0dbe652-1096-4294-a4a5-d091386d4ff1/volumes"
Jul 09 16:59:50.861017 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:50.860986    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 16:59:50.865541 ip-10-0-139-176 kubenswrapper[2586]: I0709 16:59:50.865515    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 17:00:00.143105 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143067    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"]
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143542    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="235ee757-f61a-47d1-add3-c9e780be65de" containerName="maas-api"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143562    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="235ee757-f61a-47d1-add3-c9e780be65de" containerName="maas-api"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143578    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c41c2b8-f507-4174-99ee-b44496b85870" containerName="main"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143586    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c41c2b8-f507-4174-99ee-b44496b85870" containerName="main"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143596    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0dbe652-1096-4294-a4a5-d091386d4ff1" containerName="main"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143602    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0dbe652-1096-4294-a4a5-d091386d4ff1" containerName="main"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143611    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91308378-f46f-435a-b42b-b50d8f0d4f1b" containerName="main"
Jul 09 17:00:00.143635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143617    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="91308378-f46f-435a-b42b-b50d8f0d4f1b" containerName="main"
Jul 09 17:00:00.144018 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143678    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="235ee757-f61a-47d1-add3-c9e780be65de" containerName="maas-api"
Jul 09 17:00:00.144018 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143686    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c41c2b8-f507-4174-99ee-b44496b85870" containerName="main"
Jul 09 17:00:00.144018 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143695    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0dbe652-1096-4294-a4a5-d091386d4ff1" containerName="main"
Jul 09 17:00:00.144018 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.143702    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="91308378-f46f-435a-b42b-b50d8f0d4f1b" containerName="main"
Jul 09 17:00:00.146744 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.146727    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"]
Jul 09 17:00:00.146867 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.146848    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:00:00.149319 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.149293    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 17:00:00.150030 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.150013    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 17:00:00.150145 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.150013    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h2fl4\""
Jul 09 17:00:00.150246 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.150227    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:00.157306 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.157282    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"]
Jul 09 17:00:00.159930 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.159908    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"]
Jul 09 17:00:00.246880 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.246829    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc8zc\" (UniqueName: \"kubernetes.io/projected/7c265f57-3619-4563-b67a-e8d0e8c6e1ed-kube-api-access-jc8zc\") pod \"maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt\" (UID: \"7c265f57-3619-4563-b67a-e8d0e8c6e1ed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:00.246880 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.246882    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdxt9\" (UniqueName: \"kubernetes.io/projected/6b9ee57c-4067-4431-82c9-2d3302e85e68-kube-api-access-pdxt9\") pod \"maas-api-key-cleanup-29726940-z9qsv\" (UID: \"6b9ee57c-4067-4431-82c9-2d3302e85e68\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:00:00.347928 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.347878    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc8zc\" (UniqueName: \"kubernetes.io/projected/7c265f57-3619-4563-b67a-e8d0e8c6e1ed-kube-api-access-jc8zc\") pod \"maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt\" (UID: \"7c265f57-3619-4563-b67a-e8d0e8c6e1ed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:00.347928 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.347926    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdxt9\" (UniqueName: \"kubernetes.io/projected/6b9ee57c-4067-4431-82c9-2d3302e85e68-kube-api-access-pdxt9\") pod \"maas-api-key-cleanup-29726940-z9qsv\" (UID: \"6b9ee57c-4067-4431-82c9-2d3302e85e68\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:00:00.356482 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.356454    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdxt9\" (UniqueName: \"kubernetes.io/projected/6b9ee57c-4067-4431-82c9-2d3302e85e68-kube-api-access-pdxt9\") pod \"maas-api-key-cleanup-29726940-z9qsv\" (UID: \"6b9ee57c-4067-4431-82c9-2d3302e85e68\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:00:00.356634 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.356516    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc8zc\" (UniqueName: \"kubernetes.io/projected/7c265f57-3619-4563-b67a-e8d0e8c6e1ed-kube-api-access-jc8zc\") pod \"maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt\" (UID: \"7c265f57-3619-4563-b67a-e8d0e8c6e1ed\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:00.460169 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.460078    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:00:00.468316 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.468294    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:00.610235 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.610208    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"]
Jul 09 17:00:00.612123 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:00:00.612085    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9ee57c_4067_4431_82c9_2d3302e85e68.slice/crio-8501987d26845a6b2519922a27a1683732ddc5319fe408481a13ca59c247b969 WatchSource:0}: Error finding container 8501987d26845a6b2519922a27a1683732ddc5319fe408481a13ca59c247b969: Status 404 returned error can't find the container with id 8501987d26845a6b2519922a27a1683732ddc5319fe408481a13ca59c247b969
Jul 09 17:00:00.634862 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.634839    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"]
Jul 09 17:00:00.637153 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:00:00.637129    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c265f57_3619_4563_b67a_e8d0e8c6e1ed.slice/crio-3aa2670cc859e07d9ea4766ffa2841a6f12006318d9f94e1e5a6872dc2489c5d WatchSource:0}: Error finding container 3aa2670cc859e07d9ea4766ffa2841a6f12006318d9f94e1e5a6872dc2489c5d: Status 404 returned error can't find the container with id 3aa2670cc859e07d9ea4766ffa2841a6f12006318d9f94e1e5a6872dc2489c5d
Jul 09 17:00:00.917335 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.917289    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt" event={"ID":"7c265f57-3619-4563-b67a-e8d0e8c6e1ed","Type":"ContainerStarted","Data":"3aa2670cc859e07d9ea4766ffa2841a6f12006318d9f94e1e5a6872dc2489c5d"}
Jul 09 17:00:00.918459 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:00.918424    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv" event={"ID":"6b9ee57c-4067-4431-82c9-2d3302e85e68","Type":"ContainerStarted","Data":"8501987d26845a6b2519922a27a1683732ddc5319fe408481a13ca59c247b969"}
Jul 09 17:00:02.956173 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:02.956127    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"]
Jul 09 17:00:06.326035 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:06.325998    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:00:06.950157 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:06.950122    2586 generic.go:356] "Generic (PLEG): container finished" podID="7c265f57-3619-4563-b67a-e8d0e8c6e1ed" containerID="d864c04f48d1f89df0bc2ae06806c5563d5b3a650446e0315ab7f66ec64fb493" exitCode=6
Jul 09 17:00:06.950360 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:06.950170    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt" event={"ID":"7c265f57-3619-4563-b67a-e8d0e8c6e1ed","Type":"ContainerDied","Data":"d864c04f48d1f89df0bc2ae06806c5563d5b3a650446e0315ab7f66ec64fb493"}
Jul 09 17:00:06.951829 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:06.951806    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv" event={"ID":"6b9ee57c-4067-4431-82c9-2d3302e85e68","Type":"ContainerStarted","Data":"ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278"}
Jul 09 17:00:06.979998 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:06.979951    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv" podStartSLOduration=1.629757856 podStartE2EDuration="6.979933706s" totalImagesPullingTime="5.35017585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:00:06.979272468 +0000 UTC m=+2417.905226750" watchObservedRunningTime="2026-07-09 17:00:06.979933706 +0000 UTC m=+2417.905887984"
Jul 09 17:00:07.084198 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.084172    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:07.219417 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.219376    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc8zc\" (UniqueName: \"kubernetes.io/projected/7c265f57-3619-4563-b67a-e8d0e8c6e1ed-kube-api-access-jc8zc\") pod \"7c265f57-3619-4563-b67a-e8d0e8c6e1ed\" (UID: \"7c265f57-3619-4563-b67a-e8d0e8c6e1ed\") "
Jul 09 17:00:07.221554 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.221522    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c265f57-3619-4563-b67a-e8d0e8c6e1ed-kube-api-access-jc8zc" (OuterVolumeSpecName: "kube-api-access-jc8zc") pod "7c265f57-3619-4563-b67a-e8d0e8c6e1ed" (UID: "7c265f57-3619-4563-b67a-e8d0e8c6e1ed"). InnerVolumeSpecName "kube-api-access-jc8zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:00:07.320166 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.320065    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc8zc\" (UniqueName: \"kubernetes.io/projected/7c265f57-3619-4563-b67a-e8d0e8c6e1ed-kube-api-access-jc8zc\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:07.958201 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.958160    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt" event={"ID":"7c265f57-3619-4563-b67a-e8d0e8c6e1ed","Type":"ContainerDied","Data":"3aa2670cc859e07d9ea4766ffa2841a6f12006318d9f94e1e5a6872dc2489c5d"}
Jul 09 17:00:07.958201 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.958191    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"
Jul 09 17:00:07.958735 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.958239    2586 scope.go:117] "RemoveContainer" containerID="d864c04f48d1f89df0bc2ae06806c5563d5b3a650446e0315ab7f66ec64fb493"
Jul 09 17:00:07.980391 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.980360    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"]
Jul 09 17:00:07.984070 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:07.984049    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-1876d0-29726940-2rdjt"]
Jul 09 17:00:08.775903 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:08.775866    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c265f57-3619-4563-b67a-e8d0e8c6e1ed" path="/var/lib/kubelet/pods/7c265f57-3619-4563-b67a-e8d0e8c6e1ed/volumes"
Jul 09 17:00:13.110367 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.110279    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"]
Jul 09 17:00:13.110863 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.110845    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c265f57-3619-4563-b67a-e8d0e8c6e1ed" containerName="cleanup"
Jul 09 17:00:13.110926 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.110867    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c265f57-3619-4563-b67a-e8d0e8c6e1ed" containerName="cleanup"
Jul 09 17:00:13.110980 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.110967    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c265f57-3619-4563-b67a-e8d0e8c6e1ed" containerName="cleanup"
Jul 09 17:00:13.115978 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.115958    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.118410 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.118384    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0138a7-serving-cert\""
Jul 09 17:00:13.122134 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.122106    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"]
Jul 09 17:00:13.275189 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.275147    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb67r\" (UniqueName: \"kubernetes.io/projected/a74aa210-a908-4236-8400-750a5ffcef5c-kube-api-access-sb67r\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.275372 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.275198    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a74aa210-a908-4236-8400-750a5ffcef5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.275372 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.275288    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a74aa210-a908-4236-8400-750a5ffcef5c-maas-api-tls\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.376470 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.376383    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb67r\" (UniqueName: \"kubernetes.io/projected/a74aa210-a908-4236-8400-750a5ffcef5c-kube-api-access-sb67r\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.376470 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.376431    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a74aa210-a908-4236-8400-750a5ffcef5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.376470 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.376461    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a74aa210-a908-4236-8400-750a5ffcef5c-maas-api-tls\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.377122 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.377099    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a74aa210-a908-4236-8400-750a5ffcef5c-openshift-service-ca\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.378980 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.378957    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a74aa210-a908-4236-8400-750a5ffcef5c-maas-api-tls\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.385632 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.385597    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb67r\" (UniqueName: \"kubernetes.io/projected/a74aa210-a908-4236-8400-750a5ffcef5c-kube-api-access-sb67r\") pod \"maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.427712 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.427674    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:13.567871 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.567844    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"]
Jul 09 17:00:13.569493 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:00:13.569464    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74aa210_a908_4236_8400_750a5ffcef5c.slice/crio-934a5a2fe00405524310315679f4d25619e21ed70696938746b61397f64ba252 WatchSource:0}: Error finding container 934a5a2fe00405524310315679f4d25619e21ed70696938746b61397f64ba252: Status 404 returned error can't find the container with id 934a5a2fe00405524310315679f4d25619e21ed70696938746b61397f64ba252
Jul 09 17:00:13.992966 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:13.992928    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc" event={"ID":"a74aa210-a908-4236-8400-750a5ffcef5c","Type":"ContainerStarted","Data":"934a5a2fe00405524310315679f4d25619e21ed70696938746b61397f64ba252"}
Jul 09 17:00:14.999327 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:14.999290    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc" event={"ID":"a74aa210-a908-4236-8400-750a5ffcef5c","Type":"ContainerStarted","Data":"8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572"}
Jul 09 17:00:14.999814 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:14.999365    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:15.020553 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:15.020505    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc" podStartSLOduration=0.810979941 podStartE2EDuration="2.020488678s" totalImagesPullingTime="1.209508737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:00:15.01888507 +0000 UTC m=+2425.944839351" watchObservedRunningTime="2026-07-09 17:00:15.020488678 +0000 UTC m=+2425.946442956"
Jul 09 17:00:21.011552 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.011501    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:21.332829 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.332734    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"]
Jul 09 17:00:21.341236 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.341108    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.343863 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.343832    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-254f3b-serving-cert\""
Jul 09 17:00:21.344358 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.344333    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"]
Jul 09 17:00:21.433650 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.433616    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:00:21.449469 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.449436    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c55968d3-f2a5-4bcf-803c-114ffeb06284-maas-api-tls\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.449644 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.449500    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km4fc\" (UniqueName: \"kubernetes.io/projected/c55968d3-f2a5-4bcf-803c-114ffeb06284-kube-api-access-km4fc\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.449644 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.449528    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c55968d3-f2a5-4bcf-803c-114ffeb06284-openshift-service-ca\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.550962 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.550927    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c55968d3-f2a5-4bcf-803c-114ffeb06284-openshift-service-ca\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.551158 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.551029    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c55968d3-f2a5-4bcf-803c-114ffeb06284-maas-api-tls\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.551158 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.551112    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km4fc\" (UniqueName: \"kubernetes.io/projected/c55968d3-f2a5-4bcf-803c-114ffeb06284-kube-api-access-km4fc\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.551690 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.551661    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c55968d3-f2a5-4bcf-803c-114ffeb06284-openshift-service-ca\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.553529 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.553507    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c55968d3-f2a5-4bcf-803c-114ffeb06284-maas-api-tls\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.558673 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.558645    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km4fc\" (UniqueName: \"kubernetes.io/projected/c55968d3-f2a5-4bcf-803c-114ffeb06284-kube-api-access-km4fc\") pod \"maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.656010 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.655965    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:21.807161 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:21.807136    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"]
Jul 09 17:00:21.809218 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:00:21.809183    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55968d3_f2a5_4bcf_803c_114ffeb06284.slice/crio-cecf78965d5fbd69ae4361311997ad1c83bebaabdf8d9ef3bc87fb4ed48e8f1c WatchSource:0}: Error finding container cecf78965d5fbd69ae4361311997ad1c83bebaabdf8d9ef3bc87fb4ed48e8f1c: Status 404 returned error can't find the container with id cecf78965d5fbd69ae4361311997ad1c83bebaabdf8d9ef3bc87fb4ed48e8f1c
Jul 09 17:00:22.037539 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:22.037453    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2" event={"ID":"c55968d3-f2a5-4bcf-803c-114ffeb06284","Type":"ContainerStarted","Data":"cecf78965d5fbd69ae4361311997ad1c83bebaabdf8d9ef3bc87fb4ed48e8f1c"}
Jul 09 17:00:23.043858 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:23.043823    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2" event={"ID":"c55968d3-f2a5-4bcf-803c-114ffeb06284","Type":"ContainerStarted","Data":"c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a"}
Jul 09 17:00:23.044268 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:23.043892    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:23.063696 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:23.063644    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2" podStartSLOduration=1.510320127 podStartE2EDuration="2.063626519s" totalImagesPullingTime="553.306392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:00:23.061078985 +0000 UTC m=+2433.987033253" watchObservedRunningTime="2026-07-09 17:00:23.063626519 +0000 UTC m=+2433.989580796"
Jul 09 17:00:29.055209 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:29.055180    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:34.760303 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:34.760270    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"]
Jul 09 17:00:34.760793 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:34.760526    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc" podUID="a74aa210-a908-4236-8400-750a5ffcef5c" containerName="maas-api" containerID="cri-o://8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572" gracePeriod=30
Jul 09 17:00:35.009460 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.009433    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:35.065379 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.065294    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sb67r\" (UniqueName: \"kubernetes.io/projected/a74aa210-a908-4236-8400-750a5ffcef5c-kube-api-access-sb67r\") pod \"a74aa210-a908-4236-8400-750a5ffcef5c\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") "
Jul 09 17:00:35.065379 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.065356    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a74aa210-a908-4236-8400-750a5ffcef5c-maas-api-tls\") pod \"a74aa210-a908-4236-8400-750a5ffcef5c\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") "
Jul 09 17:00:35.065566 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.065473    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a74aa210-a908-4236-8400-750a5ffcef5c-openshift-service-ca\") pod \"a74aa210-a908-4236-8400-750a5ffcef5c\" (UID: \"a74aa210-a908-4236-8400-750a5ffcef5c\") "
Jul 09 17:00:35.065816 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.065794    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a74aa210-a908-4236-8400-750a5ffcef5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a74aa210-a908-4236-8400-750a5ffcef5c" (UID: "a74aa210-a908-4236-8400-750a5ffcef5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 17:00:35.067436 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.067410    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a74aa210-a908-4236-8400-750a5ffcef5c-kube-api-access-sb67r" (OuterVolumeSpecName: "kube-api-access-sb67r") pod "a74aa210-a908-4236-8400-750a5ffcef5c" (UID: "a74aa210-a908-4236-8400-750a5ffcef5c"). InnerVolumeSpecName "kube-api-access-sb67r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:00:35.067533 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.067419    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a74aa210-a908-4236-8400-750a5ffcef5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a74aa210-a908-4236-8400-750a5ffcef5c" (UID: "a74aa210-a908-4236-8400-750a5ffcef5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:00:35.114236 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.114201    2586 generic.go:356] "Generic (PLEG): container finished" podID="a74aa210-a908-4236-8400-750a5ffcef5c" containerID="8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572" exitCode=0
Jul 09 17:00:35.114431 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.114256    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"
Jul 09 17:00:35.114431 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.114288    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc" event={"ID":"a74aa210-a908-4236-8400-750a5ffcef5c","Type":"ContainerDied","Data":"8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572"}
Jul 09 17:00:35.114431 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.114350    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc" event={"ID":"a74aa210-a908-4236-8400-750a5ffcef5c","Type":"ContainerDied","Data":"934a5a2fe00405524310315679f4d25619e21ed70696938746b61397f64ba252"}
Jul 09 17:00:35.114431 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.114383    2586 scope.go:117] "RemoveContainer" containerID="8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572"
Jul 09 17:00:35.123628 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.123610    2586 scope.go:117] "RemoveContainer" containerID="8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572"
Jul 09 17:00:35.123919 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:00:35.123900    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572\": container with ID starting with 8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572 not found: ID does not exist" containerID="8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572"
Jul 09 17:00:35.123966 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.123928    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572"} err="failed to get container status \"8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572\": rpc error: code = NotFound desc = could not find container \"8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572\": container with ID starting with 8357728fba9cadce89759bd218fb6472e29d45b2d726cf9d913f2b429aadf572 not found: ID does not exist"
Jul 09 17:00:35.135385 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.135361    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"]
Jul 09 17:00:35.138926 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.138906    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0138a7-f5d75d7f9-w2dfc"]
Jul 09 17:00:35.166268 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.166226    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a74aa210-a908-4236-8400-750a5ffcef5c-maas-api-tls\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:35.166268 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.166261    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a74aa210-a908-4236-8400-750a5ffcef5c-openshift-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:35.166268 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:35.166271    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb67r\" (UniqueName: \"kubernetes.io/projected/a74aa210-a908-4236-8400-750a5ffcef5c-kube-api-access-sb67r\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:36.772636 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:36.772602    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a74aa210-a908-4236-8400-750a5ffcef5c" path="/var/lib/kubelet/pods/a74aa210-a908-4236-8400-750a5ffcef5c/volumes"
Jul 09 17:00:37.726343 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:37.726298    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:00:41.164092 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.164061    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"]
Jul 09 17:00:41.164580 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.164255    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2" podUID="c55968d3-f2a5-4bcf-803c-114ffeb06284" containerName="maas-api" containerID="cri-o://c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a" gracePeriod=30
Jul 09 17:00:41.412822 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.412798    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:41.521667 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.521574    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c55968d3-f2a5-4bcf-803c-114ffeb06284-openshift-service-ca\") pod \"c55968d3-f2a5-4bcf-803c-114ffeb06284\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") "
Jul 09 17:00:41.521667 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.521641    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km4fc\" (UniqueName: \"kubernetes.io/projected/c55968d3-f2a5-4bcf-803c-114ffeb06284-kube-api-access-km4fc\") pod \"c55968d3-f2a5-4bcf-803c-114ffeb06284\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") "
Jul 09 17:00:41.521922 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.521756    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c55968d3-f2a5-4bcf-803c-114ffeb06284-maas-api-tls\") pod \"c55968d3-f2a5-4bcf-803c-114ffeb06284\" (UID: \"c55968d3-f2a5-4bcf-803c-114ffeb06284\") "
Jul 09 17:00:41.522085 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.522058    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c55968d3-f2a5-4bcf-803c-114ffeb06284-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c55968d3-f2a5-4bcf-803c-114ffeb06284" (UID: "c55968d3-f2a5-4bcf-803c-114ffeb06284"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 17:00:41.523774 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.523739    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c55968d3-f2a5-4bcf-803c-114ffeb06284-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c55968d3-f2a5-4bcf-803c-114ffeb06284" (UID: "c55968d3-f2a5-4bcf-803c-114ffeb06284"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:00:41.523849 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.523828    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c55968d3-f2a5-4bcf-803c-114ffeb06284-kube-api-access-km4fc" (OuterVolumeSpecName: "kube-api-access-km4fc") pod "c55968d3-f2a5-4bcf-803c-114ffeb06284" (UID: "c55968d3-f2a5-4bcf-803c-114ffeb06284"). InnerVolumeSpecName "kube-api-access-km4fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:00:41.622547 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.622508    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c55968d3-f2a5-4bcf-803c-114ffeb06284-maas-api-tls\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:41.622547 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.622543    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c55968d3-f2a5-4bcf-803c-114ffeb06284-openshift-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:41.622547 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:41.622553    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km4fc\" (UniqueName: \"kubernetes.io/projected/c55968d3-f2a5-4bcf-803c-114ffeb06284-kube-api-access-km4fc\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:00:42.151821 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.151785    2586 generic.go:356] "Generic (PLEG): container finished" podID="c55968d3-f2a5-4bcf-803c-114ffeb06284" containerID="c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a" exitCode=0
Jul 09 17:00:42.151821 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.151823    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"
Jul 09 17:00:42.152042 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.151851    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2" event={"ID":"c55968d3-f2a5-4bcf-803c-114ffeb06284","Type":"ContainerDied","Data":"c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a"}
Jul 09 17:00:42.152042 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.151908    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2" event={"ID":"c55968d3-f2a5-4bcf-803c-114ffeb06284","Type":"ContainerDied","Data":"cecf78965d5fbd69ae4361311997ad1c83bebaabdf8d9ef3bc87fb4ed48e8f1c"}
Jul 09 17:00:42.152042 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.151937    2586 scope.go:117] "RemoveContainer" containerID="c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a"
Jul 09 17:00:42.161530 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.161506    2586 scope.go:117] "RemoveContainer" containerID="c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a"
Jul 09 17:00:42.161863 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:00:42.161842    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a\": container with ID starting with c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a not found: ID does not exist" containerID="c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a"
Jul 09 17:00:42.161934 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.161874    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a"} err="failed to get container status \"c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a\": rpc error: code = NotFound desc = could not find container \"c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a\": container with ID starting with c0784520bbbcb1f6faeb7b198fbe6cee2e0c895078ebdc21b4ae660fa04e949a not found: ID does not exist"
Jul 09 17:00:42.171987 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.171961    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"]
Jul 09 17:00:42.176196 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.176169    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-254f3b-79dc6948c6-z68r2"]
Jul 09 17:00:42.773089 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:42.773059    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c55968d3-f2a5-4bcf-803c-114ffeb06284" path="/var/lib/kubelet/pods/c55968d3-f2a5-4bcf-803c-114ffeb06284/volumes"
Jul 09 17:00:51.129828 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.129794    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:00:51.419157 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419079    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"]
Jul 09 17:00:51.419424 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419413    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a74aa210-a908-4236-8400-750a5ffcef5c" containerName="maas-api"
Jul 09 17:00:51.419466 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419426    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="a74aa210-a908-4236-8400-750a5ffcef5c" containerName="maas-api"
Jul 09 17:00:51.419466 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419447    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c55968d3-f2a5-4bcf-803c-114ffeb06284" containerName="maas-api"
Jul 09 17:00:51.419466 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419453    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c55968d3-f2a5-4bcf-803c-114ffeb06284" containerName="maas-api"
Jul 09 17:00:51.419556 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419513    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c55968d3-f2a5-4bcf-803c-114ffeb06284" containerName="maas-api"
Jul 09 17:00:51.419556 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.419522    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="a74aa210-a908-4236-8400-750a5ffcef5c" containerName="maas-api"
Jul 09 17:00:51.423462 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.423446    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.425658 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.425636    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-d1f7bd-serving-cert\""
Jul 09 17:00:51.433618 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.433595    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"]
Jul 09 17:00:51.510873 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.510835    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.511063 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.510888    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.511063 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.510973    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfgk9\" (UniqueName: \"kubernetes.io/projected/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-kube-api-access-vfgk9\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.611584 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.611559    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.611684 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.611608    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.611684 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.611642    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfgk9\" (UniqueName: \"kubernetes.io/projected/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-kube-api-access-vfgk9\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.611684 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:00:51.611672    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-serving-cert: secret "maas-api-e2e-shared-a-d1f7bd-serving-cert" not found
Jul 09 17:00:51.611818 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:00:51.611742    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls podName:b9d20cb4-1405-4d6e-9c1d-8026712fd7b6 nodeName:}" failed. No retries permitted until 2026-07-09 17:00:52.111724094 +0000 UTC m=+2463.037678350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls") pod "maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" (UID: "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6") : secret "maas-api-e2e-shared-a-d1f7bd-serving-cert" not found
Jul 09 17:00:51.612256 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.612236    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:51.623311 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:51.623279    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfgk9\" (UniqueName: \"kubernetes.io/projected/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-kube-api-access-vfgk9\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:52.116664 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:52.116627    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:52.119011 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:52.118991    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls\") pod \"maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:52.334494 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:52.334459    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:52.473839 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:52.473594    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"]
Jul 09 17:00:52.475986 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:00:52.475954    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d20cb4_1405_4d6e_9c1d_8026712fd7b6.slice/crio-f0c4122729268d84015bf7a723debacccb153d8ea40247bdebd1db4e728479c3 WatchSource:0}: Error finding container f0c4122729268d84015bf7a723debacccb153d8ea40247bdebd1db4e728479c3: Status 404 returned error can't find the container with id f0c4122729268d84015bf7a723debacccb153d8ea40247bdebd1db4e728479c3
Jul 09 17:00:53.209259 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:53.209159    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" event={"ID":"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6","Type":"ContainerStarted","Data":"da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700"}
Jul 09 17:00:53.209259 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:53.209221    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" event={"ID":"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6","Type":"ContainerStarted","Data":"f0c4122729268d84015bf7a723debacccb153d8ea40247bdebd1db4e728479c3"}
Jul 09 17:00:53.209536 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:53.209265    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:53.229303 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:53.229230    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" podStartSLOduration=1.768289167 podStartE2EDuration="2.229209711s" totalImagesPullingTime="460.920544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:00:53.22885622 +0000 UTC m=+2464.154810499" watchObservedRunningTime="2026-07-09 17:00:53.229209711 +0000 UTC m=+2464.155163992"
Jul 09 17:00:59.219737 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.219702    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:00:59.512690 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.512601    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"]
Jul 09 17:00:59.516215 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.516190    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.518562 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.518540    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e779af-serving-cert\""
Jul 09 17:00:59.537412 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.537381    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"]
Jul 09 17:00:59.582699 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.582666    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm5zx\" (UniqueName: \"kubernetes.io/projected/1574c334-3609-4d1d-9a24-c3b67e491744-kube-api-access-bm5zx\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.582908 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.582707    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.582908 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.582729    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1574c334-3609-4d1d-9a24-c3b67e491744-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.683303 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.683261    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm5zx\" (UniqueName: \"kubernetes.io/projected/1574c334-3609-4d1d-9a24-c3b67e491744-kube-api-access-bm5zx\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.683511 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.683312    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.683511 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:00:59.683420    2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-serving-cert: secret "maas-api-e2e-shared-b-e779af-serving-cert" not found
Jul 09 17:00:59.683511 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.683437    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1574c334-3609-4d1d-9a24-c3b67e491744-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.683511 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:00:59.683483    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls podName:1574c334-3609-4d1d-9a24-c3b67e491744 nodeName:}" failed. No retries permitted until 2026-07-09 17:01:00.183462299 +0000 UTC m=+2471.109416556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls") pod "maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" (UID: "1574c334-3609-4d1d-9a24-c3b67e491744") : secret "maas-api-e2e-shared-b-e779af-serving-cert" not found
Jul 09 17:00:59.684043 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.684026    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1574c334-3609-4d1d-9a24-c3b67e491744-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:00:59.693093 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:00:59.693068    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm5zx\" (UniqueName: \"kubernetes.io/projected/1574c334-3609-4d1d-9a24-c3b67e491744-kube-api-access-bm5zx\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:01:00.013376 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:00.013335    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"]
Jul 09 17:01:00.013602 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:00.013573    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv" podUID="6b9ee57c-4067-4431-82c9-2d3302e85e68" containerName="cleanup" containerID="cri-o://ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278" gracePeriod=30
Jul 09 17:01:00.187527 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:00.187490    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:01:00.189926 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:00.189905    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls\") pod \"maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:01:00.426919 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:00.426867    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:01:00.562264 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:00.562231    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"]
Jul 09 17:01:00.564254 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:01:00.564225    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1574c334_3609_4d1d_9a24_c3b67e491744.slice/crio-30b96700074e4d6c2cd8dd8b312e603b96d3f4cb57ba022ddaa439356f5e4265 WatchSource:0}: Error finding container 30b96700074e4d6c2cd8dd8b312e603b96d3f4cb57ba022ddaa439356f5e4265: Status 404 returned error can't find the container with id 30b96700074e4d6c2cd8dd8b312e603b96d3f4cb57ba022ddaa439356f5e4265
Jul 09 17:01:01.251270 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:01.251235    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" event={"ID":"1574c334-3609-4d1d-9a24-c3b67e491744","Type":"ContainerStarted","Data":"30b96700074e4d6c2cd8dd8b312e603b96d3f4cb57ba022ddaa439356f5e4265"}
Jul 09 17:01:02.258937 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:02.258898    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" event={"ID":"1574c334-3609-4d1d-9a24-c3b67e491744","Type":"ContainerStarted","Data":"5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9"}
Jul 09 17:01:02.259344 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:02.258999    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:01:02.281681 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:02.281626    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" podStartSLOduration=2.59134645 podStartE2EDuration="3.281608125s" totalImagesPullingTime="690.261675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:00:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:01:02.279503237 +0000 UTC m=+2473.205457515" watchObservedRunningTime="2026-07-09 17:01:02.281608125 +0000 UTC m=+2473.207562402"
Jul 09 17:01:02.846427 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:02.846391    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:01:08.271528 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:08.271492    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:01:15.831261 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.831226    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"]
Jul 09 17:01:15.836090 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.836064    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:15.839511 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.839489    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"auth-test-model-d1f7bd-kserve-self-signed-certs\""
Jul 09 17:01:15.839660 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.839525    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"default-dockercfg-pg5x8\""
Jul 09 17:01:15.839660 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.839525    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"openshift-service-ca.crt\""
Jul 09 17:01:15.840123 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.840108    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"kube-root-ca.crt\""
Jul 09 17:01:15.851471 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.851449    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"]
Jul 09 17:01:15.929727 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.929690    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-home\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:15.929916 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.929810    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-model-cache\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:15.929916 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.929852    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692272fe-be5a-4320-b910-88ddf596b915-tls-certs\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:15.929991 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.929942    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-dshm\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:15.930032 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.929991    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vwtk\" (UniqueName: \"kubernetes.io/projected/692272fe-be5a-4320-b910-88ddf596b915-kube-api-access-5vwtk\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:15.930032 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:15.930015    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-tmp-dir\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.030939 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.030902    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-model-cache\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.030939 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.030944    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692272fe-be5a-4320-b910-88ddf596b915-tls-certs\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031263 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.030989    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-dshm\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031263 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.031013    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vwtk\" (UniqueName: \"kubernetes.io/projected/692272fe-be5a-4320-b910-88ddf596b915-kube-api-access-5vwtk\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031263 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.031034    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-tmp-dir\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031263 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.031066    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-home\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031457 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.031310    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-model-cache\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031457 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.031442    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-tmp-dir\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.031587 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.031482    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-home\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.033249 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.033225    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-dshm\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.033521 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.033503    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692272fe-be5a-4320-b910-88ddf596b915-tls-certs\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.040880 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.040854    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vwtk\" (UniqueName: \"kubernetes.io/projected/692272fe-be5a-4320-b910-88ddf596b915-kube-api-access-5vwtk\") pod \"auth-test-model-d1f7bd-kserve-896dc899f-kjl2s\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") " pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.145009 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.144971    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:16.287807 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.287780    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"]
Jul 09 17:01:16.289220 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:01:16.289197    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod692272fe_be5a_4320_b910_88ddf596b915.slice/crio-80ca945b810419b8cff785a87828940371cf6a0665f794754d3433fe3fa5599d WatchSource:0}: Error finding container 80ca945b810419b8cff785a87828940371cf6a0665f794754d3433fe3fa5599d: Status 404 returned error can't find the container with id 80ca945b810419b8cff785a87828940371cf6a0665f794754d3433fe3fa5599d
Jul 09 17:01:16.337540 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:16.337500    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s" event={"ID":"692272fe-be5a-4320-b910-88ddf596b915","Type":"ContainerStarted","Data":"80ca945b810419b8cff785a87828940371cf6a0665f794754d3433fe3fa5599d"}
Jul 09 17:01:17.343893 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:17.343856    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s" event={"ID":"692272fe-be5a-4320-b910-88ddf596b915","Type":"ContainerStarted","Data":"dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c"}
Jul 09 17:01:17.371679 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:17.371627    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s" podStartSLOduration=2.371608239 podStartE2EDuration="2.371608239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:01:17.369904045 +0000 UTC m=+2488.295858323" watchObservedRunningTime="2026-07-09 17:01:17.371608239 +0000 UTC m=+2488.297562517"
Jul 09 17:01:19.337381 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:19.337341    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:01:26.145473 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:26.145435    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:26.146017 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:26.145521    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:26.158347 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:26.158307    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:26.401743 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:26.401657    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:01:30.205383 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.205357    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:01:30.266912 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.266869    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdxt9\" (UniqueName: \"kubernetes.io/projected/6b9ee57c-4067-4431-82c9-2d3302e85e68-kube-api-access-pdxt9\") pod \"6b9ee57c-4067-4431-82c9-2d3302e85e68\" (UID: \"6b9ee57c-4067-4431-82c9-2d3302e85e68\") "
Jul 09 17:01:30.269014 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.268934    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b9ee57c-4067-4431-82c9-2d3302e85e68-kube-api-access-pdxt9" (OuterVolumeSpecName: "kube-api-access-pdxt9") pod "6b9ee57c-4067-4431-82c9-2d3302e85e68" (UID: "6b9ee57c-4067-4431-82c9-2d3302e85e68"). InnerVolumeSpecName "kube-api-access-pdxt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:01:30.367813 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.367753    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdxt9\" (UniqueName: \"kubernetes.io/projected/6b9ee57c-4067-4431-82c9-2d3302e85e68-kube-api-access-pdxt9\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:01:30.411638 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.411597    2586 generic.go:356] "Generic (PLEG): container finished" podID="6b9ee57c-4067-4431-82c9-2d3302e85e68" containerID="ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278" exitCode=137
Jul 09 17:01:30.411815 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.411652    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv" event={"ID":"6b9ee57c-4067-4431-82c9-2d3302e85e68","Type":"ContainerDied","Data":"ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278"}
Jul 09 17:01:30.411815 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.411690    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv" event={"ID":"6b9ee57c-4067-4431-82c9-2d3302e85e68","Type":"ContainerDied","Data":"8501987d26845a6b2519922a27a1683732ddc5319fe408481a13ca59c247b969"}
Jul 09 17:01:30.411815 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.411712    2586 scope.go:117] "RemoveContainer" containerID="ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278"
Jul 09 17:01:30.411815 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.411651    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"
Jul 09 17:01:30.427521 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.427498    2586 scope.go:117] "RemoveContainer" containerID="ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278"
Jul 09 17:01:30.427837 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:01:30.427818    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278\": container with ID starting with ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278 not found: ID does not exist" containerID="ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278"
Jul 09 17:01:30.427921 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.427843    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278"} err="failed to get container status \"ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278\": rpc error: code = NotFound desc = could not find container \"ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278\": container with ID starting with ebea497a329ecfaccc9f2f75cf6be4fb4e754efff88dd3bcd2aa48d8f0725278 not found: ID does not exist"
Jul 09 17:01:30.442007 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.441979    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"]
Jul 09 17:01:30.445659 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.445633    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726940-z9qsv"]
Jul 09 17:01:30.776047 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:30.776014    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9ee57c-4067-4431-82c9-2d3302e85e68" path="/var/lib/kubelet/pods/6b9ee57c-4067-4431-82c9-2d3302e85e68/volumes"
Jul 09 17:01:59.847549 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.847517    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"]
Jul 09 17:01:59.848014 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.847908    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9ee57c-4067-4431-82c9-2d3302e85e68" containerName="cleanup"
Jul 09 17:01:59.848014 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.847921    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9ee57c-4067-4431-82c9-2d3302e85e68" containerName="cleanup"
Jul 09 17:01:59.848014 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.847984    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9ee57c-4067-4431-82c9-2d3302e85e68" containerName="cleanup"
Jul 09 17:01:59.850999 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.850982    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:01:59.853410 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.853387    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"openshift-service-ca.crt\""
Jul 09 17:01:59.854336 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.854312    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"default-dockercfg-5tb6x\""
Jul 09 17:01:59.854458 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.854426    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"auth-test-model-e779af-kserve-self-signed-certs\""
Jul 09 17:01:59.854506 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.854481    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"kube-root-ca.crt\""
Jul 09 17:01:59.865046 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.865022    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"]
Jul 09 17:01:59.939817 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.939749    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c96nh\" (UniqueName: \"kubernetes.io/projected/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-kube-api-access-c96nh\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:01:59.939996 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.939844    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-dshm\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:01:59.939996 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.939884    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tls-certs\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:01:59.939996 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.939954    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tmp-dir\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:01:59.940179 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.940046    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-model-cache\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:01:59.940179 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:01:59.940108    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-home\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.040930 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.040892    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-home\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041153 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.040947    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c96nh\" (UniqueName: \"kubernetes.io/projected/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-kube-api-access-c96nh\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041153 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.040976    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-dshm\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041153 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.040998    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tls-certs\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041153 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.041033    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tmp-dir\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041153 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.041075    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-model-cache\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041430 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.041382    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-home\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041519 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.041495    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tmp-dir\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.041556 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.041507    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-model-cache\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.043380 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.043358    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-dshm\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.043532 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.043512    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tls-certs\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.052218 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.052191    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c96nh\" (UniqueName: \"kubernetes.io/projected/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-kube-api-access-c96nh\") pod \"auth-test-model-e779af-kserve-5ccd575c7f-tk2kv\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") " pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.160800 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.160680    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:00.311529 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.311503    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"]
Jul 09 17:02:00.312517 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:02:00.312488    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24960bd5_9512_4fe4_9f70_466d0ff9e3ff.slice/crio-258edceca7a8ed9cd4ebc01e386bfff51844f1c8c9fcdff02a9d1ba06314a7d6 WatchSource:0}: Error finding container 258edceca7a8ed9cd4ebc01e386bfff51844f1c8c9fcdff02a9d1ba06314a7d6: Status 404 returned error can't find the container with id 258edceca7a8ed9cd4ebc01e386bfff51844f1c8c9fcdff02a9d1ba06314a7d6
Jul 09 17:02:00.567877 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.567752    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv" event={"ID":"24960bd5-9512-4fe4-9f70-466d0ff9e3ff","Type":"ContainerStarted","Data":"e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827"}
Jul 09 17:02:00.567877 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.567820    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv" event={"ID":"24960bd5-9512-4fe4-9f70-466d0ff9e3ff","Type":"ContainerStarted","Data":"258edceca7a8ed9cd4ebc01e386bfff51844f1c8c9fcdff02a9d1ba06314a7d6"}
Jul 09 17:02:00.593448 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:00.593387    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv" podStartSLOduration=1.5933631099999999 podStartE2EDuration="1.59336311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:02:00.592410776 +0000 UTC m=+2531.518365055" watchObservedRunningTime="2026-07-09 17:02:00.59336311 +0000 UTC m=+2531.519317390"
Jul 09 17:02:02.146540 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:02.146501    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:02:10.160962 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:10.160903    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:10.160962 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:10.160972    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:10.173638 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:10.173610    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:10.628941 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:10.628914    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:02:16.643769 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:16.643721    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:02:37.333588 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:37.333544    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:02:56.141982 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:02:56.141945    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:03:13.348703 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:03:13.348613    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:03:31.837374 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:03:31.837338    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:03:51.825486 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:03:51.825452    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:04:06.339223 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:06.339187    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"]
Jul 09 17:04:07.333892 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:07.333858    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"]
Jul 09 17:04:08.428975 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:08.428933    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:04:12.619229 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.619192    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"]
Jul 09 17:04:12.622620 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.622601    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.625178 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.625153    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"sub-test-model-d1f7bd-kserve-self-signed-certs\""
Jul 09 17:04:12.633057 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.633032    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"]
Jul 09 17:04:12.734081 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.734038    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-home\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.734267 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.734106    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-dshm\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.734267 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.734147    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-model-cache\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.734267 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.734164    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-tmp-dir\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.734267 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.734182    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab97bffe-4807-486b-bb29-c8e89bbc1562-tls-certs\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.734267 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.734250    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qhnz\" (UniqueName: \"kubernetes.io/projected/ab97bffe-4807-486b-bb29-c8e89bbc1562-kube-api-access-9qhnz\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.835769 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.835727    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qhnz\" (UniqueName: \"kubernetes.io/projected/ab97bffe-4807-486b-bb29-c8e89bbc1562-kube-api-access-9qhnz\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.835968 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.835822    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-home\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.835968 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.835857    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-dshm\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.835968 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.835877    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-model-cache\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.835968 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.835893    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-tmp-dir\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.835968 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.835908    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab97bffe-4807-486b-bb29-c8e89bbc1562-tls-certs\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.836273 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.836250    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-home\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.836373 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.836352    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-model-cache\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.836427 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.836379    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-tmp-dir\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.838175 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.838150    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-dshm\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.838579 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.838559    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab97bffe-4807-486b-bb29-c8e89bbc1562-tls-certs\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.844265 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.844240    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qhnz\" (UniqueName: \"kubernetes.io/projected/ab97bffe-4807-486b-bb29-c8e89bbc1562-kube-api-access-9qhnz\") pod \"sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") " pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:12.933042 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:12.932948    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:13.067197 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:13.067171    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"]
Jul 09 17:04:13.068476 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:04:13.068449    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab97bffe_4807_486b_bb29_c8e89bbc1562.slice/crio-ecd99441532e6fc4303513482d20766e62db125fbdfc88bc99ccd9c3b6e838a0 WatchSource:0}: Error finding container ecd99441532e6fc4303513482d20766e62db125fbdfc88bc99ccd9c3b6e838a0: Status 404 returned error can't find the container with id ecd99441532e6fc4303513482d20766e62db125fbdfc88bc99ccd9c3b6e838a0
Jul 09 17:04:13.257250 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:13.257164    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl" event={"ID":"ab97bffe-4807-486b-bb29-c8e89bbc1562","Type":"ContainerStarted","Data":"23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677"}
Jul 09 17:04:13.257250 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:13.257207    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl" event={"ID":"ab97bffe-4807-486b-bb29-c8e89bbc1562","Type":"ContainerStarted","Data":"ecd99441532e6fc4303513482d20766e62db125fbdfc88bc99ccd9c3b6e838a0"}
Jul 09 17:04:13.276054 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:13.276002    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl" podStartSLOduration=1.275985459 podStartE2EDuration="1.275985459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:04:13.273460856 +0000 UTC m=+2664.199415136" watchObservedRunningTime="2026-07-09 17:04:13.275985459 +0000 UTC m=+2664.201939749"
Jul 09 17:04:21.359531 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.359489    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s" podUID="692272fe-be5a-4320-b910-88ddf596b915" containerName="main" containerID="cri-o://dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c" gracePeriod=45
Jul 09 17:04:21.610568 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.610496    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:04:21.707926 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.707886    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-tmp-dir\") pod \"692272fe-be5a-4320-b910-88ddf596b915\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") "
Jul 09 17:04:21.708096 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.707939    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692272fe-be5a-4320-b910-88ddf596b915-tls-certs\") pod \"692272fe-be5a-4320-b910-88ddf596b915\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") "
Jul 09 17:04:21.708096 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.707971    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vwtk\" (UniqueName: \"kubernetes.io/projected/692272fe-be5a-4320-b910-88ddf596b915-kube-api-access-5vwtk\") pod \"692272fe-be5a-4320-b910-88ddf596b915\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") "
Jul 09 17:04:21.708096 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.707995    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-dshm\") pod \"692272fe-be5a-4320-b910-88ddf596b915\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") "
Jul 09 17:04:21.708096 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708018    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-home\") pod \"692272fe-be5a-4320-b910-88ddf596b915\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") "
Jul 09 17:04:21.708096 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708043    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-model-cache\") pod \"692272fe-be5a-4320-b910-88ddf596b915\" (UID: \"692272fe-be5a-4320-b910-88ddf596b915\") "
Jul 09 17:04:21.708386 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708182    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "692272fe-be5a-4320-b910-88ddf596b915" (UID: "692272fe-be5a-4320-b910-88ddf596b915"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:21.708386 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708335    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-home" (OuterVolumeSpecName: "home") pod "692272fe-be5a-4320-b910-88ddf596b915" (UID: "692272fe-be5a-4320-b910-88ddf596b915"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:21.708386 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708367    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-model-cache" (OuterVolumeSpecName: "model-cache") pod "692272fe-be5a-4320-b910-88ddf596b915" (UID: "692272fe-be5a-4320-b910-88ddf596b915"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:21.708502 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708431    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:21.708502 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.708446    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:21.710357 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.710323    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/692272fe-be5a-4320-b910-88ddf596b915-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "692272fe-be5a-4320-b910-88ddf596b915" (UID: "692272fe-be5a-4320-b910-88ddf596b915"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:04:21.710460 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.710383    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692272fe-be5a-4320-b910-88ddf596b915-kube-api-access-5vwtk" (OuterVolumeSpecName: "kube-api-access-5vwtk") pod "692272fe-be5a-4320-b910-88ddf596b915" (UID: "692272fe-be5a-4320-b910-88ddf596b915"). InnerVolumeSpecName "kube-api-access-5vwtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:04:21.710460 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.710393    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-dshm" (OuterVolumeSpecName: "dshm") pod "692272fe-be5a-4320-b910-88ddf596b915" (UID: "692272fe-be5a-4320-b910-88ddf596b915"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:21.809863 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.809725    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:21.809863 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.809792    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/692272fe-be5a-4320-b910-88ddf596b915-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:21.809863 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.809807    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/692272fe-be5a-4320-b910-88ddf596b915-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:21.809863 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:21.809816    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vwtk\" (UniqueName: \"kubernetes.io/projected/692272fe-be5a-4320-b910-88ddf596b915-kube-api-access-5vwtk\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.306953 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.306916    2586 generic.go:356] "Generic (PLEG): container finished" podID="692272fe-be5a-4320-b910-88ddf596b915" containerID="dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c" exitCode=0
Jul 09 17:04:22.307165 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.306962    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"
Jul 09 17:04:22.307165 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.306993    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s" event={"ID":"692272fe-be5a-4320-b910-88ddf596b915","Type":"ContainerDied","Data":"dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c"}
Jul 09 17:04:22.307165 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.307032    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s" event={"ID":"692272fe-be5a-4320-b910-88ddf596b915","Type":"ContainerDied","Data":"80ca945b810419b8cff785a87828940371cf6a0665f794754d3433fe3fa5599d"}
Jul 09 17:04:22.307165 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.307055    2586 scope.go:117] "RemoveContainer" containerID="dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c"
Jul 09 17:04:22.316752 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.316733    2586 scope.go:117] "RemoveContainer" containerID="dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c"
Jul 09 17:04:22.317081 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:04:22.317059    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c\": container with ID starting with dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c not found: ID does not exist" containerID="dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c"
Jul 09 17:04:22.317176 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.317087    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c"} err="failed to get container status \"dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c\": rpc error: code = NotFound desc = could not find container \"dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c\": container with ID starting with dea149197c9807a1f0ce9087d4e3b570c36884820dcf991413b9c23bd9d6745c not found: ID does not exist"
Jul 09 17:04:22.327967 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.327944    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"]
Jul 09 17:04:22.331507 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.331485    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/auth-test-model-d1f7bd-kserve-896dc899f-kjl2s"]
Jul 09 17:04:22.354609 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.354571    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv" podUID="24960bd5-9512-4fe4-9f70-466d0ff9e3ff" containerName="main" containerID="cri-o://e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827" gracePeriod=45
Jul 09 17:04:22.605168 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.605141    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:04:22.617876 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.617851    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tls-certs\") pod \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") "
Jul 09 17:04:22.618024 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.617897    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tmp-dir\") pod \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") "
Jul 09 17:04:22.618024 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.617924    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c96nh\" (UniqueName: \"kubernetes.io/projected/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-kube-api-access-c96nh\") pod \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") "
Jul 09 17:04:22.618024 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.617945    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-home\") pod \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") "
Jul 09 17:04:22.618024 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.617989    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-dshm\") pod \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") "
Jul 09 17:04:22.618240 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.618030    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-model-cache\") pod \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\" (UID: \"24960bd5-9512-4fe4-9f70-466d0ff9e3ff\") "
Jul 09 17:04:22.618240 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.618160    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "24960bd5-9512-4fe4-9f70-466d0ff9e3ff" (UID: "24960bd5-9512-4fe4-9f70-466d0ff9e3ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:22.618360 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.618237    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-home" (OuterVolumeSpecName: "home") pod "24960bd5-9512-4fe4-9f70-466d0ff9e3ff" (UID: "24960bd5-9512-4fe4-9f70-466d0ff9e3ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:22.618360 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.618299    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.618441 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.618384    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "24960bd5-9512-4fe4-9f70-466d0ff9e3ff" (UID: "24960bd5-9512-4fe4-9f70-466d0ff9e3ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:22.620120 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.620097    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "24960bd5-9512-4fe4-9f70-466d0ff9e3ff" (UID: "24960bd5-9512-4fe4-9f70-466d0ff9e3ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:04:22.620644 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.620627    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-dshm" (OuterVolumeSpecName: "dshm") pod "24960bd5-9512-4fe4-9f70-466d0ff9e3ff" (UID: "24960bd5-9512-4fe4-9f70-466d0ff9e3ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:04:22.620730 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.620652    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-kube-api-access-c96nh" (OuterVolumeSpecName: "kube-api-access-c96nh") pod "24960bd5-9512-4fe4-9f70-466d0ff9e3ff" (UID: "24960bd5-9512-4fe4-9f70-466d0ff9e3ff"). InnerVolumeSpecName "kube-api-access-c96nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:04:22.719695 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.719657    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c96nh\" (UniqueName: \"kubernetes.io/projected/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-kube-api-access-c96nh\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.719695 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.719689    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.719695 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.719698    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.719695 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.719706    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.720002 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.719716    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24960bd5-9512-4fe4-9f70-466d0ff9e3ff-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:04:22.773196 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.773162    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="692272fe-be5a-4320-b910-88ddf596b915" path="/var/lib/kubelet/pods/692272fe-be5a-4320-b910-88ddf596b915/volumes"
Jul 09 17:04:22.934204 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.934103    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:22.934204 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.934160    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:22.946488 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:22.946461    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:23.313897 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.313795    2586 generic.go:356] "Generic (PLEG): container finished" podID="24960bd5-9512-4fe4-9f70-466d0ff9e3ff" containerID="e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827" exitCode=0
Jul 09 17:04:23.313897 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.313844    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"
Jul 09 17:04:23.314142 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.313894    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv" event={"ID":"24960bd5-9512-4fe4-9f70-466d0ff9e3ff","Type":"ContainerDied","Data":"e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827"}
Jul 09 17:04:23.314142 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.313945    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv" event={"ID":"24960bd5-9512-4fe4-9f70-466d0ff9e3ff","Type":"ContainerDied","Data":"258edceca7a8ed9cd4ebc01e386bfff51844f1c8c9fcdff02a9d1ba06314a7d6"}
Jul 09 17:04:23.314142 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.313969    2586 scope.go:117] "RemoveContainer" containerID="e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827"
Jul 09 17:04:23.323288 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.323266    2586 scope.go:117] "RemoveContainer" containerID="e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827"
Jul 09 17:04:23.323598 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:04:23.323574    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827\": container with ID starting with e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827 not found: ID does not exist" containerID="e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827"
Jul 09 17:04:23.323688 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.323605    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827"} err="failed to get container status \"e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827\": rpc error: code = NotFound desc = could not find container \"e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827\": container with ID starting with e5ca816d04830153975b31cbcc3640e0cbcb13679866e52f1bfbf2cf583e4827 not found: ID does not exist"
Jul 09 17:04:23.326748 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.326727    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:04:23.336650 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.336625    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"]
Jul 09 17:04:23.338990 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:23.338970    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/auth-test-model-e779af-kserve-5ccd575c7f-tk2kv"]
Jul 09 17:04:24.772933 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:24.772898    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24960bd5-9512-4fe4-9f70-466d0ff9e3ff" path="/var/lib/kubelet/pods/24960bd5-9512-4fe4-9f70-466d0ff9e3ff/volumes"
Jul 09 17:04:27.825987 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:27.825951    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:04:41.837715 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.837627    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"]
Jul 09 17:04:41.838182 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.838030    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="692272fe-be5a-4320-b910-88ddf596b915" containerName="main"
Jul 09 17:04:41.838182 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.838044    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="692272fe-be5a-4320-b910-88ddf596b915" containerName="main"
Jul 09 17:04:41.838182 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.838060    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24960bd5-9512-4fe4-9f70-466d0ff9e3ff" containerName="main"
Jul 09 17:04:41.838182 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.838066    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="24960bd5-9512-4fe4-9f70-466d0ff9e3ff" containerName="main"
Jul 09 17:04:41.838182 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.838136    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="24960bd5-9512-4fe4-9f70-466d0ff9e3ff" containerName="main"
Jul 09 17:04:41.838182 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.838145    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="692272fe-be5a-4320-b910-88ddf596b915" containerName="main"
Jul 09 17:04:41.847206 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.847178    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:41.850064 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.850039    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"openshift-service-ca.crt\""
Jul 09 17:04:41.850725 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.850700    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"sub-test-model-e779af-kserve-self-signed-certs\""
Jul 09 17:04:41.850725 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.850713    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"default-dockercfg-5tb6x\""
Jul 09 17:04:41.851028 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.851011    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"kube-root-ca.crt\""
Jul 09 17:04:41.851722 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.851702    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"]
Jul 09 17:04:41.986272 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.986236    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-home\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:41.986461 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.986281    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-dshm\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:41.986461 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.986368    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tls-certs\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:41.986461 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.986404    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-model-cache\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:41.986461 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.986442    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tmp-dir\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:41.986600 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:41.986461    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xldrc\" (UniqueName: \"kubernetes.io/projected/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-kube-api-access-xldrc\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087181 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087145    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-dshm\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087376 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087202    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tls-certs\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087376 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087220    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-model-cache\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087376 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087246    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tmp-dir\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087376 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xldrc\" (UniqueName: \"kubernetes.io/projected/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-kube-api-access-xldrc\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087561 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087413    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-home\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087825 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087735    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tmp-dir\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087825 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087786    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-model-cache\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.087965 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.087845    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-home\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.089546 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.089527    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-dshm\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.089874 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.089856    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tls-certs\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.095317 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.095293    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xldrc\" (UniqueName: \"kubernetes.io/projected/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-kube-api-access-xldrc\") pod \"sub-test-model-e779af-kserve-69fd76df89-trfv2\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") " pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.158924 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.158885    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:42.291042 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.291014    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"]
Jul 09 17:04:42.293319 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:04:42.293294    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode132b86b_92ba_47a5_a36b_a1f38fedc1d9.slice/crio-0d816367a42cbd286224f5760dcff6b27a1ee5e461d854b84e18b4c39aa15b81 WatchSource:0}: Error finding container 0d816367a42cbd286224f5760dcff6b27a1ee5e461d854b84e18b4c39aa15b81: Status 404 returned error can't find the container with id 0d816367a42cbd286224f5760dcff6b27a1ee5e461d854b84e18b4c39aa15b81
Jul 09 17:04:42.419064 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.419025    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" event={"ID":"e132b86b-92ba-47a5-a36b-a1f38fedc1d9","Type":"ContainerStarted","Data":"32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69"}
Jul 09 17:04:42.419262 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.419081    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" event={"ID":"e132b86b-92ba-47a5-a36b-a1f38fedc1d9","Type":"ContainerStarted","Data":"0d816367a42cbd286224f5760dcff6b27a1ee5e461d854b84e18b4c39aa15b81"}
Jul 09 17:04:42.436293 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:42.436234    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" podStartSLOduration=1.436216413 podStartE2EDuration="1.436216413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:04:42.433664365 +0000 UTC m=+2693.359618654" watchObservedRunningTime="2026-07-09 17:04:42.436216413 +0000 UTC m=+2693.362170692"
Jul 09 17:04:46.331327 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:46.331289    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:04:50.902952 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:50.902920    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 17:04:50.906565 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:50.906541    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6qf5t_49361ed9-3456-4c1e-aff8-68a5c23fbcbe/ovn-acl-logging/0.log"
Jul 09 17:04:52.159313 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:52.159278    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:52.159714 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:52.159325    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:52.171942 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:52.171908    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:04:52.485744 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:04:52.485656    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:05:01.436971 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:01.436928    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:05:21.822112 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:21.822073    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:05:26.424078 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:26.424039    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"]
Jul 09 17:05:27.317301 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:27.317267    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"]
Jul 09 17:05:39.831075 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:39.831029    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:05:41.444216 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.444174    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl" podUID="ab97bffe-4807-486b-bb29-c8e89bbc1562" containerName="main" containerID="cri-o://23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677" gracePeriod=45
Jul 09 17:05:41.691531 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.691504    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:05:41.738886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.738801    2586 generic.go:356] "Generic (PLEG): container finished" podID="ab97bffe-4807-486b-bb29-c8e89bbc1562" containerID="23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677" exitCode=0
Jul 09 17:05:41.738886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.738845    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"
Jul 09 17:05:41.738886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.738877    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl" event={"ID":"ab97bffe-4807-486b-bb29-c8e89bbc1562","Type":"ContainerDied","Data":"23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677"}
Jul 09 17:05:41.739128 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.738922    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl" event={"ID":"ab97bffe-4807-486b-bb29-c8e89bbc1562","Type":"ContainerDied","Data":"ecd99441532e6fc4303513482d20766e62db125fbdfc88bc99ccd9c3b6e838a0"}
Jul 09 17:05:41.739128 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.738935    2586 scope.go:117] "RemoveContainer" containerID="23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677"
Jul 09 17:05:41.747727 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.747710    2586 scope.go:117] "RemoveContainer" containerID="23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677"
Jul 09 17:05:41.748011 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:05:41.747992    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677\": container with ID starting with 23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677 not found: ID does not exist" containerID="23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677"
Jul 09 17:05:41.748093 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.748019    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677"} err="failed to get container status \"23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677\": rpc error: code = NotFound desc = could not find container \"23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677\": container with ID starting with 23e3a4457b313a27e302ffff0056aacf54c72d4f0a5b8faf297e37dbd0e76677 not found: ID does not exist"
Jul 09 17:05:41.840270 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840234    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-model-cache\") pod \"ab97bffe-4807-486b-bb29-c8e89bbc1562\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") "
Jul 09 17:05:41.840416 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840308    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-home\") pod \"ab97bffe-4807-486b-bb29-c8e89bbc1562\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") "
Jul 09 17:05:41.840416 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840351    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-dshm\") pod \"ab97bffe-4807-486b-bb29-c8e89bbc1562\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") "
Jul 09 17:05:41.840416 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840380    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qhnz\" (UniqueName: \"kubernetes.io/projected/ab97bffe-4807-486b-bb29-c8e89bbc1562-kube-api-access-9qhnz\") pod \"ab97bffe-4807-486b-bb29-c8e89bbc1562\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") "
Jul 09 17:05:41.840416 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840411    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-tmp-dir\") pod \"ab97bffe-4807-486b-bb29-c8e89bbc1562\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") "
Jul 09 17:05:41.840602 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840476    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab97bffe-4807-486b-bb29-c8e89bbc1562-tls-certs\") pod \"ab97bffe-4807-486b-bb29-c8e89bbc1562\" (UID: \"ab97bffe-4807-486b-bb29-c8e89bbc1562\") "
Jul 09 17:05:41.840602 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840565    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-model-cache" (OuterVolumeSpecName: "model-cache") pod "ab97bffe-4807-486b-bb29-c8e89bbc1562" (UID: "ab97bffe-4807-486b-bb29-c8e89bbc1562"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:41.840602 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840581    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-home" (OuterVolumeSpecName: "home") pod "ab97bffe-4807-486b-bb29-c8e89bbc1562" (UID: "ab97bffe-4807-486b-bb29-c8e89bbc1562"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:41.840793 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840685    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ab97bffe-4807-486b-bb29-c8e89bbc1562" (UID: "ab97bffe-4807-486b-bb29-c8e89bbc1562"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:41.840793 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840788    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:41.840910 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840804    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:41.840910 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.840812    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:41.842440 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.842418    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-dshm" (OuterVolumeSpecName: "dshm") pod "ab97bffe-4807-486b-bb29-c8e89bbc1562" (UID: "ab97bffe-4807-486b-bb29-c8e89bbc1562"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:41.842526 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.842476    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab97bffe-4807-486b-bb29-c8e89bbc1562-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ab97bffe-4807-486b-bb29-c8e89bbc1562" (UID: "ab97bffe-4807-486b-bb29-c8e89bbc1562"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:05:41.842563 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.842530    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab97bffe-4807-486b-bb29-c8e89bbc1562-kube-api-access-9qhnz" (OuterVolumeSpecName: "kube-api-access-9qhnz") pod "ab97bffe-4807-486b-bb29-c8e89bbc1562" (UID: "ab97bffe-4807-486b-bb29-c8e89bbc1562"). InnerVolumeSpecName "kube-api-access-9qhnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:05:41.941338 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.941300    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab97bffe-4807-486b-bb29-c8e89bbc1562-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:41.941338 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.941330    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qhnz\" (UniqueName: \"kubernetes.io/projected/ab97bffe-4807-486b-bb29-c8e89bbc1562-kube-api-access-9qhnz\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:41.941338 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:41.941340    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab97bffe-4807-486b-bb29-c8e89bbc1562-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.060299 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.060268    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"]
Jul 09 17:05:42.064416 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.064389    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/sub-test-model-d1f7bd-kserve-7ffb747b99-vlxcl"]
Jul 09 17:05:42.338583 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.338543    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" podUID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" containerName="main" containerID="cri-o://32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69" gracePeriod=45
Jul 09 17:05:42.476011 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.475960    2586 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" podUID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": dial tcp 10.134.0.56:8000: connect: connection refused"
Jul 09 17:05:42.592665 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.592603    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:05:42.648836 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.648800    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-dshm\") pod \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") "
Jul 09 17:05:42.648998 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.648852    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tmp-dir\") pod \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") "
Jul 09 17:05:42.648998 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.648888    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tls-certs\") pod \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") "
Jul 09 17:05:42.648998 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.648915    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xldrc\" (UniqueName: \"kubernetes.io/projected/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-kube-api-access-xldrc\") pod \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") "
Jul 09 17:05:42.648998 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.648961    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-model-cache\") pod \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") "
Jul 09 17:05:42.649196 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.649091    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-home\") pod \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\" (UID: \"e132b86b-92ba-47a5-a36b-a1f38fedc1d9\") "
Jul 09 17:05:42.649196 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.649156    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "e132b86b-92ba-47a5-a36b-a1f38fedc1d9" (UID: "e132b86b-92ba-47a5-a36b-a1f38fedc1d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:42.649196 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.649167    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e132b86b-92ba-47a5-a36b-a1f38fedc1d9" (UID: "e132b86b-92ba-47a5-a36b-a1f38fedc1d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:42.649370 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.649354    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-home" (OuterVolumeSpecName: "home") pod "e132b86b-92ba-47a5-a36b-a1f38fedc1d9" (UID: "e132b86b-92ba-47a5-a36b-a1f38fedc1d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:42.649426 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.649411    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.649474 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.649431    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.651005 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.650983    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-kube-api-access-xldrc" (OuterVolumeSpecName: "kube-api-access-xldrc") pod "e132b86b-92ba-47a5-a36b-a1f38fedc1d9" (UID: "e132b86b-92ba-47a5-a36b-a1f38fedc1d9"). InnerVolumeSpecName "kube-api-access-xldrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:05:42.651428 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.651405    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-dshm" (OuterVolumeSpecName: "dshm") pod "e132b86b-92ba-47a5-a36b-a1f38fedc1d9" (UID: "e132b86b-92ba-47a5-a36b-a1f38fedc1d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:05:42.651470 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.651415    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e132b86b-92ba-47a5-a36b-a1f38fedc1d9" (UID: "e132b86b-92ba-47a5-a36b-a1f38fedc1d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:05:42.746471 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.746432    2586 generic.go:356] "Generic (PLEG): container finished" podID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" containerID="32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69" exitCode=0
Jul 09 17:05:42.746666 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.746476    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"
Jul 09 17:05:42.746666 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.746516    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" event={"ID":"e132b86b-92ba-47a5-a36b-a1f38fedc1d9","Type":"ContainerDied","Data":"32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69"}
Jul 09 17:05:42.746666 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.746558    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2" event={"ID":"e132b86b-92ba-47a5-a36b-a1f38fedc1d9","Type":"ContainerDied","Data":"0d816367a42cbd286224f5760dcff6b27a1ee5e461d854b84e18b4c39aa15b81"}
Jul 09 17:05:42.746666 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.746580    2586 scope.go:117] "RemoveContainer" containerID="32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69"
Jul 09 17:05:42.750066 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.750042    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.750066 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.750065    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xldrc\" (UniqueName: \"kubernetes.io/projected/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-kube-api-access-xldrc\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.750226 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.750074    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.750226 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.750083    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e132b86b-92ba-47a5-a36b-a1f38fedc1d9-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:05:42.756321 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.756302    2586 scope.go:117] "RemoveContainer" containerID="32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69"
Jul 09 17:05:42.756579 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:05:42.756560    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69\": container with ID starting with 32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69 not found: ID does not exist" containerID="32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69"
Jul 09 17:05:42.756634 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.756586    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69"} err="failed to get container status \"32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69\": rpc error: code = NotFound desc = could not find container \"32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69\": container with ID starting with 32e95788958d17236085d0f9a50013c9d983b725f8128eeb8cc01ecf38776c69 not found: ID does not exist"
Jul 09 17:05:42.779438 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.779392    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab97bffe-4807-486b-bb29-c8e89bbc1562" path="/var/lib/kubelet/pods/ab97bffe-4807-486b-bb29-c8e89bbc1562/volumes"
Jul 09 17:05:42.779886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.779867    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"]
Jul 09 17:05:42.779931 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:42.779913    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/sub-test-model-e779af-kserve-69fd76df89-trfv2"]
Jul 09 17:05:44.771787 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:44.771731    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" path="/var/lib/kubelet/pods/e132b86b-92ba-47a5-a36b-a1f38fedc1d9/volumes"
Jul 09 17:05:56.432713 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.432676    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"]
Jul 09 17:05:56.434278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.433699    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" containerName="main"
Jul 09 17:05:56.434278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.433731    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" containerName="main"
Jul 09 17:05:56.434278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.433789    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab97bffe-4807-486b-bb29-c8e89bbc1562" containerName="main"
Jul 09 17:05:56.434278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.433800    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab97bffe-4807-486b-bb29-c8e89bbc1562" containerName="main"
Jul 09 17:05:56.434278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.434021    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab97bffe-4807-486b-bb29-c8e89bbc1562" containerName="main"
Jul 09 17:05:56.434278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.434034    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e132b86b-92ba-47a5-a36b-a1f38fedc1d9" containerName="main"
Jul 09 17:05:56.439578 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.439549    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.443424 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.443386    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"rate-test-model-d1f7bd-kserve-self-signed-certs\""
Jul 09 17:05:56.443569 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.443391    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"default-dockercfg-pg5x8\""
Jul 09 17:05:56.443569 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.443476    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"openshift-service-ca.crt\""
Jul 09 17:05:56.443833 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.443817    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d1f7bd\"/\"kube-root-ca.crt\""
Jul 09 17:05:56.445192 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.445171    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"]
Jul 09 17:05:56.578060 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.578014    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nftm9\" (UniqueName: \"kubernetes.io/projected/9d94abbe-6bae-4637-baa2-6bc89c22942d-kube-api-access-nftm9\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.578060 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.578055    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d94abbe-6bae-4637-baa2-6bc89c22942d-tls-certs\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.578299 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.578083    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-tmp-dir\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.578299 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.578135    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-home\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.578299 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.578163    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-dshm\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.578299 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.578246    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-model-cache\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.679481 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.679443    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nftm9\" (UniqueName: \"kubernetes.io/projected/9d94abbe-6bae-4637-baa2-6bc89c22942d-kube-api-access-nftm9\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.679481 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.679482    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d94abbe-6bae-4637-baa2-6bc89c22942d-tls-certs\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.679721 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.679511    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-tmp-dir\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.679721 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.679547    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-home\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.679721 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.679587    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-dshm\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.679721 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.679634    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-model-cache\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.680084 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.680054    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-tmp-dir\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.680084 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.680073    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-home\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.680257 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.680117    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-model-cache\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.681880 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.681859    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-dshm\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.682041 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.682022    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d94abbe-6bae-4637-baa2-6bc89c22942d-tls-certs\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.688058 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.687995    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nftm9\" (UniqueName: \"kubernetes.io/projected/9d94abbe-6bae-4637-baa2-6bc89c22942d-kube-api-access-nftm9\") pod \"rate-test-model-d1f7bd-kserve-ccd6499-wz59g\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") " pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.751310 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.751274    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:05:56.891058 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:56.891023    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"]
Jul 09 17:05:56.892487 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:05:56.892460    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d94abbe_6bae_4637_baa2_6bc89c22942d.slice/crio-e8051a2cc8cd74d64927ca589a5a8ceb752408625590d6a4ca94a4253ac94071 WatchSource:0}: Error finding container e8051a2cc8cd74d64927ca589a5a8ceb752408625590d6a4ca94a4253ac94071: Status 404 returned error can't find the container with id e8051a2cc8cd74d64927ca589a5a8ceb752408625590d6a4ca94a4253ac94071
Jul 09 17:05:57.829071 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:57.829028    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g" event={"ID":"9d94abbe-6bae-4637-baa2-6bc89c22942d","Type":"ContainerStarted","Data":"ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805"}
Jul 09 17:05:57.829468 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:57.829081    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g" event={"ID":"9d94abbe-6bae-4637-baa2-6bc89c22942d","Type":"ContainerStarted","Data":"e8051a2cc8cd74d64927ca589a5a8ceb752408625590d6a4ca94a4253ac94071"}
Jul 09 17:05:57.846748 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:05:57.846690    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g" podStartSLOduration=1.846668666 podStartE2EDuration="1.846668666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:05:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:05:57.845664758 +0000 UTC m=+2768.771619048" watchObservedRunningTime="2026-07-09 17:05:57.846668666 +0000 UTC m=+2768.772622948"
Jul 09 17:06:01.136441 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:01.136401    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:06:06.751562 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:06.751522    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:06:06.752117 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:06.751603    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:06:06.764155 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:06.764133    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:06:06.889717 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:06.889683    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:06:15.926010 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.925927    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"]
Jul 09 17:06:15.929466 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.929449    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:15.931732 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.931709    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"rate-test-model-e779af-kserve-self-signed-certs\""
Jul 09 17:06:15.932825 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.932806    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"default-dockercfg-5tb6x\""
Jul 09 17:06:15.932931 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.932829    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"openshift-service-ca.crt\""
Jul 09 17:06:15.932931 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.932896    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e779af\"/\"kube-root-ca.crt\""
Jul 09 17:06:15.945045 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:15.945009    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"]
Jul 09 17:06:16.048485 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.048446    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g67zc\" (UniqueName: \"kubernetes.io/projected/d83ece0b-b15e-4688-8a76-7a125be90796-kube-api-access-g67zc\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.048661 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.048502    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-tmp-dir\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.048661 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.048555    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d83ece0b-b15e-4688-8a76-7a125be90796-tls-certs\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.048661 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.048571    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-dshm\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.048661 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.048595    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-model-cache\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.048843 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.048695    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-home\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150136 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150091    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-model-cache\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150136 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150143    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-home\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150396 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150198    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g67zc\" (UniqueName: \"kubernetes.io/projected/d83ece0b-b15e-4688-8a76-7a125be90796-kube-api-access-g67zc\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150396 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150252    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-tmp-dir\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150396 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150298    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d83ece0b-b15e-4688-8a76-7a125be90796-tls-certs\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150396 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150329    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-dshm\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150605 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150576    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-model-cache\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150605 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150598    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-home\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.150710 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.150638    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-tmp-dir\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.152586 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.152564    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-dshm\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.152987 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.152963    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d83ece0b-b15e-4688-8a76-7a125be90796-tls-certs\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.157879 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.157856    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g67zc\" (UniqueName: \"kubernetes.io/projected/d83ece0b-b15e-4688-8a76-7a125be90796-kube-api-access-g67zc\") pod \"rate-test-model-e779af-kserve-d45cc7b59-r5bkr\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") " pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.245357 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.245264    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:16.382864 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.382837    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"]
Jul 09 17:06:16.384951 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:06:16.384921    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd83ece0b_b15e_4688_8a76_7a125be90796.slice/crio-a2490a174a89df6a9f7b13788b08700cd8ce362192302eb40d94506ef5ef4d9d WatchSource:0}: Error finding container a2490a174a89df6a9f7b13788b08700cd8ce362192302eb40d94506ef5ef4d9d: Status 404 returned error can't find the container with id a2490a174a89df6a9f7b13788b08700cd8ce362192302eb40d94506ef5ef4d9d
Jul 09 17:06:16.933634 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.933598    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr" event={"ID":"d83ece0b-b15e-4688-8a76-7a125be90796","Type":"ContainerStarted","Data":"6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d"}
Jul 09 17:06:16.934063 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.933647    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr" event={"ID":"d83ece0b-b15e-4688-8a76-7a125be90796","Type":"ContainerStarted","Data":"a2490a174a89df6a9f7b13788b08700cd8ce362192302eb40d94506ef5ef4d9d"}
Jul 09 17:06:16.952100 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:16.952052    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr" podStartSLOduration=1.952030994 podStartE2EDuration="1.952030994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:06:16.95090976 +0000 UTC m=+2787.876864037" watchObservedRunningTime="2026-07-09 17:06:16.952030994 +0000 UTC m=+2787.877985271"
Jul 09 17:06:21.236936 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:21.236873    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:06:26.246012 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:26.245965    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:26.246594 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:26.246050    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:26.258670 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:26.258643    2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:27.000116 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:27.000089    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:06:36.931958 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:36.931923    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:06:52.425533 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:52.425494    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:06:58.734206 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:58.734163    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"]
Jul 09 17:06:59.609626 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:06:59.609586    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"]
Jul 09 17:07:08.427174 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:08.427129    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:07:13.756605 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:13.756541    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g" podUID="9d94abbe-6bae-4637-baa2-6bc89c22942d" containerName="main" containerID="cri-o://ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805" gracePeriod=45
Jul 09 17:07:14.014676 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.014608    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:07:14.160371 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160339    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d94abbe-6bae-4637-baa2-6bc89c22942d-tls-certs\") pod \"9d94abbe-6bae-4637-baa2-6bc89c22942d\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") "
Jul 09 17:07:14.160582 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160416    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-model-cache\") pod \"9d94abbe-6bae-4637-baa2-6bc89c22942d\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") "
Jul 09 17:07:14.160582 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160522    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-tmp-dir\") pod \"9d94abbe-6bae-4637-baa2-6bc89c22942d\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") "
Jul 09 17:07:14.160582 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160569    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-dshm\") pod \"9d94abbe-6bae-4637-baa2-6bc89c22942d\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") "
Jul 09 17:07:14.160836 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160597    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nftm9\" (UniqueName: \"kubernetes.io/projected/9d94abbe-6bae-4637-baa2-6bc89c22942d-kube-api-access-nftm9\") pod \"9d94abbe-6bae-4637-baa2-6bc89c22942d\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") "
Jul 09 17:07:14.160836 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160619    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-home\") pod \"9d94abbe-6bae-4637-baa2-6bc89c22942d\" (UID: \"9d94abbe-6bae-4637-baa2-6bc89c22942d\") "
Jul 09 17:07:14.160836 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160663    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d94abbe-6bae-4637-baa2-6bc89c22942d" (UID: "9d94abbe-6bae-4637-baa2-6bc89c22942d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.160836 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160806    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d94abbe-6bae-4637-baa2-6bc89c22942d" (UID: "9d94abbe-6bae-4637-baa2-6bc89c22942d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.160993 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160920    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-home" (OuterVolumeSpecName: "home") pod "9d94abbe-6bae-4637-baa2-6bc89c22942d" (UID: "9d94abbe-6bae-4637-baa2-6bc89c22942d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.161031 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.160996    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:14.161031 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.161015    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:14.161031 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.161028    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:14.162573 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.162548    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d94abbe-6bae-4637-baa2-6bc89c22942d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d94abbe-6bae-4637-baa2-6bc89c22942d" (UID: "9d94abbe-6bae-4637-baa2-6bc89c22942d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:07:14.162682 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.162588    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-dshm" (OuterVolumeSpecName: "dshm") pod "9d94abbe-6bae-4637-baa2-6bc89c22942d" (UID: "9d94abbe-6bae-4637-baa2-6bc89c22942d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.162682 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.162595    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d94abbe-6bae-4637-baa2-6bc89c22942d-kube-api-access-nftm9" (OuterVolumeSpecName: "kube-api-access-nftm9") pod "9d94abbe-6bae-4637-baa2-6bc89c22942d" (UID: "9d94abbe-6bae-4637-baa2-6bc89c22942d"). InnerVolumeSpecName "kube-api-access-nftm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:14.238666 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.238626    2586 generic.go:356] "Generic (PLEG): container finished" podID="9d94abbe-6bae-4637-baa2-6bc89c22942d" containerID="ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805" exitCode=0
Jul 09 17:07:14.238666 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.238667    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"
Jul 09 17:07:14.238948 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.238675    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g" event={"ID":"9d94abbe-6bae-4637-baa2-6bc89c22942d","Type":"ContainerDied","Data":"ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805"}
Jul 09 17:07:14.238948 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.238710    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g" event={"ID":"9d94abbe-6bae-4637-baa2-6bc89c22942d","Type":"ContainerDied","Data":"e8051a2cc8cd74d64927ca589a5a8ceb752408625590d6a4ca94a4253ac94071"}
Jul 09 17:07:14.238948 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.238729    2586 scope.go:117] "RemoveContainer" containerID="ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805"
Jul 09 17:07:14.248269 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.248246    2586 scope.go:117] "RemoveContainer" containerID="ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805"
Jul 09 17:07:14.248535 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:07:14.248515    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805\": container with ID starting with ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805 not found: ID does not exist" containerID="ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805"
Jul 09 17:07:14.248607 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.248543    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805"} err="failed to get container status \"ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805\": rpc error: code = NotFound desc = could not find container \"ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805\": container with ID starting with ce2585fd78aec422fc0dd32fe3d48602a6401cc5b2af0a6f8be41d676da22805 not found: ID does not exist"
Jul 09 17:07:14.258653 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.258627    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"]
Jul 09 17:07:14.261635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.261614    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nftm9\" (UniqueName: \"kubernetes.io/projected/9d94abbe-6bae-4637-baa2-6bc89c22942d-kube-api-access-nftm9\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:14.261635 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.261636    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d94abbe-6bae-4637-baa2-6bc89c22942d-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:14.261813 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.261647    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d94abbe-6bae-4637-baa2-6bc89c22942d-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:14.262389 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.262367    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d1f7bd/rate-test-model-d1f7bd-kserve-ccd6499-wz59g"]
Jul 09 17:07:14.630354 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.630285    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr" podUID="d83ece0b-b15e-4688-8a76-7a125be90796" containerName="main" containerID="cri-o://6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d" gracePeriod=45
Jul 09 17:07:14.773004 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.772974    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d94abbe-6bae-4637-baa2-6bc89c22942d" path="/var/lib/kubelet/pods/9d94abbe-6bae-4637-baa2-6bc89c22942d/volumes"
Jul 09 17:07:14.895175 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.895146    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:07:14.968562 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.968520    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d83ece0b-b15e-4688-8a76-7a125be90796-tls-certs\") pod \"d83ece0b-b15e-4688-8a76-7a125be90796\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") "
Jul 09 17:07:14.968783 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.968591    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-dshm\") pod \"d83ece0b-b15e-4688-8a76-7a125be90796\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") "
Jul 09 17:07:14.968783 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.968636    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-model-cache\") pod \"d83ece0b-b15e-4688-8a76-7a125be90796\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") "
Jul 09 17:07:14.968783 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.968700    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-tmp-dir\") pod \"d83ece0b-b15e-4688-8a76-7a125be90796\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") "
Jul 09 17:07:14.968783 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.968732    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g67zc\" (UniqueName: \"kubernetes.io/projected/d83ece0b-b15e-4688-8a76-7a125be90796-kube-api-access-g67zc\") pod \"d83ece0b-b15e-4688-8a76-7a125be90796\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") "
Jul 09 17:07:14.969001 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.968805    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-home\") pod \"d83ece0b-b15e-4688-8a76-7a125be90796\" (UID: \"d83ece0b-b15e-4688-8a76-7a125be90796\") "
Jul 09 17:07:14.969052 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.969005    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d83ece0b-b15e-4688-8a76-7a125be90796" (UID: "d83ece0b-b15e-4688-8a76-7a125be90796"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.969052 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.969011    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-model-cache" (OuterVolumeSpecName: "model-cache") pod "d83ece0b-b15e-4688-8a76-7a125be90796" (UID: "d83ece0b-b15e-4688-8a76-7a125be90796"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.969270 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.969251    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-home" (OuterVolumeSpecName: "home") pod "d83ece0b-b15e-4688-8a76-7a125be90796" (UID: "d83ece0b-b15e-4688-8a76-7a125be90796"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.970900 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.970880    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d83ece0b-b15e-4688-8a76-7a125be90796-kube-api-access-g67zc" (OuterVolumeSpecName: "kube-api-access-g67zc") pod "d83ece0b-b15e-4688-8a76-7a125be90796" (UID: "d83ece0b-b15e-4688-8a76-7a125be90796"). InnerVolumeSpecName "kube-api-access-g67zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:14.971265 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.971247    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-dshm" (OuterVolumeSpecName: "dshm") pod "d83ece0b-b15e-4688-8a76-7a125be90796" (UID: "d83ece0b-b15e-4688-8a76-7a125be90796"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 17:07:14.971265 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:14.971255    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d83ece0b-b15e-4688-8a76-7a125be90796-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d83ece0b-b15e-4688-8a76-7a125be90796" (UID: "d83ece0b-b15e-4688-8a76-7a125be90796"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:07:15.069848 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.069798    2586 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-dshm\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:15.069848 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.069838    2586 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-model-cache\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:15.069848 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.069850    2586 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-tmp-dir\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:15.069848 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.069858    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g67zc\" (UniqueName: \"kubernetes.io/projected/d83ece0b-b15e-4688-8a76-7a125be90796-kube-api-access-g67zc\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:15.069848 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.069866    2586 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d83ece0b-b15e-4688-8a76-7a125be90796-home\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:15.070140 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.069875    2586 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d83ece0b-b15e-4688-8a76-7a125be90796-tls-certs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:15.246831 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.246729    2586 generic.go:356] "Generic (PLEG): container finished" podID="d83ece0b-b15e-4688-8a76-7a125be90796" containerID="6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d" exitCode=0
Jul 09 17:07:15.246831 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.246797    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"
Jul 09 17:07:15.246831 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.246819    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr" event={"ID":"d83ece0b-b15e-4688-8a76-7a125be90796","Type":"ContainerDied","Data":"6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d"}
Jul 09 17:07:15.247056 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.246866    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr" event={"ID":"d83ece0b-b15e-4688-8a76-7a125be90796","Type":"ContainerDied","Data":"a2490a174a89df6a9f7b13788b08700cd8ce362192302eb40d94506ef5ef4d9d"}
Jul 09 17:07:15.247056 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.246889    2586 scope.go:117] "RemoveContainer" containerID="6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d"
Jul 09 17:07:15.256693 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.256663    2586 scope.go:117] "RemoveContainer" containerID="6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d"
Jul 09 17:07:15.256970 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:07:15.256950    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d\": container with ID starting with 6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d not found: ID does not exist" containerID="6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d"
Jul 09 17:07:15.257026 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.256980    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d"} err="failed to get container status \"6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d\": rpc error: code = NotFound desc = could not find container \"6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d\": container with ID starting with 6044d3fc92ceffa6f31ac368932595cbe31d56627ab628ccd8633d0183ca043d not found: ID does not exist"
Jul 09 17:07:15.269957 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.269931    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"]
Jul 09 17:07:15.273665 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:15.273642    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e779af/rate-test-model-e779af-kserve-d45cc7b59-r5bkr"]
Jul 09 17:07:16.778301 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:16.778268    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d83ece0b-b15e-4688-8a76-7a125be90796" path="/var/lib/kubelet/pods/d83ece0b-b15e-4688-8a76-7a125be90796/volumes"
Jul 09 17:07:22.137018 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.136984    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7234-140209238729776"]
Jul 09 17:07:22.137501 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.137345    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d94abbe-6bae-4637-baa2-6bc89c22942d" containerName="main"
Jul 09 17:07:22.137501 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.137356    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d94abbe-6bae-4637-baa2-6bc89c22942d" containerName="main"
Jul 09 17:07:22.137501 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.137364    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d83ece0b-b15e-4688-8a76-7a125be90796" containerName="main"
Jul 09 17:07:22.137501 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.137370    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="d83ece0b-b15e-4688-8a76-7a125be90796" containerName="main"
Jul 09 17:07:22.137501 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.137431    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d94abbe-6bae-4637-baa2-6bc89c22942d" containerName="main"
Jul 09 17:07:22.137501 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.137442    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="d83ece0b-b15e-4688-8a76-7a125be90796" containerName="main"
Jul 09 17:07:22.141522 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.141504    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:22.143899 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.143875    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-6ttj4\""
Jul 09 17:07:22.149209 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.149013    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238729776"]
Jul 09 17:07:22.231417 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.231382    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q7kq\" (UniqueName: \"kubernetes.io/projected/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3-kube-api-access-7q7kq\") pod \"test-curl-7234-140209238729776\" (UID: \"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3\") " pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:22.332687 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.332643    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q7kq\" (UniqueName: \"kubernetes.io/projected/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3-kube-api-access-7q7kq\") pod \"test-curl-7234-140209238729776\" (UID: \"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3\") " pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:22.341136 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.341109    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q7kq\" (UniqueName: \"kubernetes.io/projected/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3-kube-api-access-7q7kq\") pod \"test-curl-7234-140209238729776\" (UID: \"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3\") " pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:22.452514 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.452394    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:22.590817 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.587453    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238729776"]
Jul 09 17:07:22.590817 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:07:22.590607    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb86f14a6_8c5b_4b7c_923a_c5a850fb5ec3.slice/crio-4d5f27283bcb8607123a965905f32aedd7bf7ad8ca420b42000e4ad097be74a3 WatchSource:0}: Error finding container 4d5f27283bcb8607123a965905f32aedd7bf7ad8ca420b42000e4ad097be74a3: Status 404 returned error can't find the container with id 4d5f27283bcb8607123a965905f32aedd7bf7ad8ca420b42000e4ad097be74a3
Jul 09 17:07:22.592351 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:22.592334    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 17:07:23.290105 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:23.290073    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238729776" event={"ID":"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3","Type":"ContainerStarted","Data":"4d5f27283bcb8607123a965905f32aedd7bf7ad8ca420b42000e4ad097be74a3"}
Jul 09 17:07:24.034264 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:24.034231    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:07:24.296297 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:24.296206    2586 generic.go:356] "Generic (PLEG): container finished" podID="b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" containerID="3b3706db06d77871a02faa7b2c945c8b1c35e9ea9175595b65e2400f2c205773" exitCode=0
Jul 09 17:07:24.296727 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:24.296297    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238729776" event={"ID":"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3","Type":"ContainerDied","Data":"3b3706db06d77871a02faa7b2c945c8b1c35e9ea9175595b65e2400f2c205773"}
Jul 09 17:07:25.437376 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.437346    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:25.469600 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.469573    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7234-140209238729776_b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3/test-curl-7234-140209238729776/0.log"
Jul 09 17:07:25.491170 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.491134    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7234-140209238729776"]
Jul 09 17:07:25.494418 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.494386    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7234-140209238729776"]
Jul 09 17:07:25.563812 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.563712    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q7kq\" (UniqueName: \"kubernetes.io/projected/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3-kube-api-access-7q7kq\") pod \"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3\" (UID: \"b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3\") "
Jul 09 17:07:25.565976 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.565942    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3-kube-api-access-7q7kq" (OuterVolumeSpecName: "kube-api-access-7q7kq") pod "b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" (UID: "b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3"). InnerVolumeSpecName "kube-api-access-7q7kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:25.622096 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.622057    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7234-140209238730416"]
Jul 09 17:07:25.622446 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.622432    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" containerName="test-curl-7234-140209238729776"
Jul 09 17:07:25.622505 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.622448    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" containerName="test-curl-7234-140209238729776"
Jul 09 17:07:25.622543 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.622509    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" containerName="test-curl-7234-140209238729776"
Jul 09 17:07:25.625504 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.625488    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:25.629373 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.629349    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238730416"]
Jul 09 17:07:25.665291 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.665251    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q7kq\" (UniqueName: \"kubernetes.io/projected/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3-kube-api-access-7q7kq\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:25.766606 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.766575    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vj7z\" (UniqueName: \"kubernetes.io/projected/112644ee-e484-43c0-890e-6a16c9375eea-kube-api-access-4vj7z\") pod \"test-curl-7234-140209238730416\" (UID: \"112644ee-e484-43c0-890e-6a16c9375eea\") " pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:25.867203 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.867166    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vj7z\" (UniqueName: \"kubernetes.io/projected/112644ee-e484-43c0-890e-6a16c9375eea-kube-api-access-4vj7z\") pod \"test-curl-7234-140209238730416\" (UID: \"112644ee-e484-43c0-890e-6a16c9375eea\") " pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:25.875862 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.875832    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vj7z\" (UniqueName: \"kubernetes.io/projected/112644ee-e484-43c0-890e-6a16c9375eea-kube-api-access-4vj7z\") pod \"test-curl-7234-140209238730416\" (UID: \"112644ee-e484-43c0-890e-6a16c9375eea\") " pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:25.935046 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:25.935007    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:26.066697 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:26.066668    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238730416"]
Jul 09 17:07:26.068516 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:07:26.068490    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod112644ee_e484_43c0_890e_6a16c9375eea.slice/crio-f8f12110b92d97e8a9042e0f2dbba65662deacb9c269dd8206f8e34b5631b264 WatchSource:0}: Error finding container f8f12110b92d97e8a9042e0f2dbba65662deacb9c269dd8206f8e34b5631b264: Status 404 returned error can't find the container with id f8f12110b92d97e8a9042e0f2dbba65662deacb9c269dd8206f8e34b5631b264
Jul 09 17:07:26.306920 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:26.306888    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:26.307076 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:26.306894    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d5f27283bcb8607123a965905f32aedd7bf7ad8ca420b42000e4ad097be74a3"
Jul 09 17:07:26.308199 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:26.308170    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238730416" event={"ID":"112644ee-e484-43c0-890e-6a16c9375eea","Type":"ContainerStarted","Data":"f8f12110b92d97e8a9042e0f2dbba65662deacb9c269dd8206f8e34b5631b264"}
Jul 09 17:07:26.317512 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:07:26.317487    2586 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7234-140209238729776\" is forbidden: User \"system:node:ip-10-0-139-176.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-139-176.ec2.internal' and this object" podUID="b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" pod="openshift-ingress/test-curl-7234-140209238729776"
Jul 09 17:07:26.772777 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:26.772684    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3" path="/var/lib/kubelet/pods/b86f14a6-8c5b-4b7c-923a-c5a850fb5ec3/volumes"
Jul 09 17:07:27.314714 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:27.314684    2586 generic.go:356] "Generic (PLEG): container finished" podID="112644ee-e484-43c0-890e-6a16c9375eea" containerID="4fbee49812b433944fe7a6465bb0bccab36659eeaec0d9e7ce295635e3db787e" exitCode=0
Jul 09 17:07:27.314920 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:27.314784    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238730416" event={"ID":"112644ee-e484-43c0-890e-6a16c9375eea","Type":"ContainerDied","Data":"4fbee49812b433944fe7a6465bb0bccab36659eeaec0d9e7ce295635e3db787e"}
Jul 09 17:07:28.462871 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.462839    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:28.482534 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.482496    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7234-140209238730416_112644ee-e484-43c0-890e-6a16c9375eea/test-curl-7234-140209238730416/0.log"
Jul 09 17:07:28.503245 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.503203    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7234-140209238730416"]
Jul 09 17:07:28.507434 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.507408    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7234-140209238730416"]
Jul 09 17:07:28.587442 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.587401    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vj7z\" (UniqueName: \"kubernetes.io/projected/112644ee-e484-43c0-890e-6a16c9375eea-kube-api-access-4vj7z\") pod \"112644ee-e484-43c0-890e-6a16c9375eea\" (UID: \"112644ee-e484-43c0-890e-6a16c9375eea\") "
Jul 09 17:07:28.589431 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.589406    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/112644ee-e484-43c0-890e-6a16c9375eea-kube-api-access-4vj7z" (OuterVolumeSpecName: "kube-api-access-4vj7z") pod "112644ee-e484-43c0-890e-6a16c9375eea" (UID: "112644ee-e484-43c0-890e-6a16c9375eea"). InnerVolumeSpecName "kube-api-access-4vj7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:28.656846 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.656809    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:28.657202 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.657190    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="112644ee-e484-43c0-890e-6a16c9375eea" containerName="test-curl-7234-140209238730416"
Jul 09 17:07:28.657246 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.657203    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="112644ee-e484-43c0-890e-6a16c9375eea" containerName="test-curl-7234-140209238730416"
Jul 09 17:07:28.657278 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.657274    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="112644ee-e484-43c0-890e-6a16c9375eea" containerName="test-curl-7234-140209238730416"
Jul 09 17:07:28.660246 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.660229    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:28.667004 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.666971    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:28.688033 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.688003    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vj7z\" (UniqueName: \"kubernetes.io/projected/112644ee-e484-43c0-890e-6a16c9375eea-kube-api-access-4vj7z\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:28.773269 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.773233    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="112644ee-e484-43c0-890e-6a16c9375eea" path="/var/lib/kubelet/pods/112644ee-e484-43c0-890e-6a16c9375eea/volumes"
Jul 09 17:07:28.789425 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.789396    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28fd\" (UniqueName: \"kubernetes.io/projected/5326316d-c956-4a0f-b0f8-039be01d6509-kube-api-access-l28fd\") pod \"test-curl-7234-140209238695984\" (UID: \"5326316d-c956-4a0f-b0f8-039be01d6509\") " pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:28.890605 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.890502    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28fd\" (UniqueName: \"kubernetes.io/projected/5326316d-c956-4a0f-b0f8-039be01d6509-kube-api-access-l28fd\") pod \"test-curl-7234-140209238695984\" (UID: \"5326316d-c956-4a0f-b0f8-039be01d6509\") " pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:28.899036 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.899002    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28fd\" (UniqueName: \"kubernetes.io/projected/5326316d-c956-4a0f-b0f8-039be01d6509-kube-api-access-l28fd\") pod \"test-curl-7234-140209238695984\" (UID: \"5326316d-c956-4a0f-b0f8-039be01d6509\") " pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:28.970755 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:28.970715    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:29.099893 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:29.099865    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:29.101347 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:07:29.101325    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5326316d_c956_4a0f_b0f8_039be01d6509.slice/crio-cb8a998a7dc8296d08409d3ac7d7a71da071fbd062ac46d9cfc60a0e068b34fc WatchSource:0}: Error finding container cb8a998a7dc8296d08409d3ac7d7a71da071fbd062ac46d9cfc60a0e068b34fc: Status 404 returned error can't find the container with id cb8a998a7dc8296d08409d3ac7d7a71da071fbd062ac46d9cfc60a0e068b34fc
Jul 09 17:07:29.326550 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:29.326514    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238695984" event={"ID":"5326316d-c956-4a0f-b0f8-039be01d6509","Type":"ContainerStarted","Data":"cb8a998a7dc8296d08409d3ac7d7a71da071fbd062ac46d9cfc60a0e068b34fc"}
Jul 09 17:07:29.327802 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:29.327772    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238730416"
Jul 09 17:07:29.327946 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:29.327808    2586 scope.go:117] "RemoveContainer" containerID="4fbee49812b433944fe7a6465bb0bccab36659eeaec0d9e7ce295635e3db787e"
Jul 09 17:07:30.334550 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:30.334519    2586 generic.go:356] "Generic (PLEG): container finished" podID="5326316d-c956-4a0f-b0f8-039be01d6509" containerID="5c42eba62c464e6c11e759723f2c61b9d61240d9fc2140adbf7044356a31fd70" exitCode=0
Jul 09 17:07:30.335041 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:30.334604    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238695984" event={"ID":"5326316d-c956-4a0f-b0f8-039be01d6509","Type":"ContainerDied","Data":"5c42eba62c464e6c11e759723f2c61b9d61240d9fc2140adbf7044356a31fd70"}
Jul 09 17:07:31.480619 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.480592    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:31.500408 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.500373    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7234-140209238695984_5326316d-c956-4a0f-b0f8-039be01d6509/test-curl-7234-140209238695984/0.log"
Jul 09 17:07:31.523486 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.523454    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:31.529124 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.529101    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:31.616013 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.615915    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l28fd\" (UniqueName: \"kubernetes.io/projected/5326316d-c956-4a0f-b0f8-039be01d6509-kube-api-access-l28fd\") pod \"5326316d-c956-4a0f-b0f8-039be01d6509\" (UID: \"5326316d-c956-4a0f-b0f8-039be01d6509\") "
Jul 09 17:07:31.618002 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.617978    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5326316d-c956-4a0f-b0f8-039be01d6509-kube-api-access-l28fd" (OuterVolumeSpecName: "kube-api-access-l28fd") pod "5326316d-c956-4a0f-b0f8-039be01d6509" (UID: "5326316d-c956-4a0f-b0f8-039be01d6509"). InnerVolumeSpecName "kube-api-access-l28fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:31.646756 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.646723    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:31.647105 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.647093    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5326316d-c956-4a0f-b0f8-039be01d6509" containerName="test-curl-7234-140209238695984"
Jul 09 17:07:31.647148 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.647107    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="5326316d-c956-4a0f-b0f8-039be01d6509" containerName="test-curl-7234-140209238695984"
Jul 09 17:07:31.647183 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.647175    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="5326316d-c956-4a0f-b0f8-039be01d6509" containerName="test-curl-7234-140209238695984"
Jul 09 17:07:31.650094 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.650076    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:31.655004 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.654982    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:31.716677 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.716641    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l28fd\" (UniqueName: \"kubernetes.io/projected/5326316d-c956-4a0f-b0f8-039be01d6509-kube-api-access-l28fd\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:31.817253 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.817214    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mslp2\" (UniqueName: \"kubernetes.io/projected/d072d97a-b151-4c30-8ed5-ffe48b50817d-kube-api-access-mslp2\") pod \"test-curl-7234-140209238695984\" (UID: \"d072d97a-b151-4c30-8ed5-ffe48b50817d\") " pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:31.918594 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.918512    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mslp2\" (UniqueName: \"kubernetes.io/projected/d072d97a-b151-4c30-8ed5-ffe48b50817d-kube-api-access-mslp2\") pod \"test-curl-7234-140209238695984\" (UID: \"d072d97a-b151-4c30-8ed5-ffe48b50817d\") " pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:31.927426 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.927397    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mslp2\" (UniqueName: \"kubernetes.io/projected/d072d97a-b151-4c30-8ed5-ffe48b50817d-kube-api-access-mslp2\") pod \"test-curl-7234-140209238695984\" (UID: \"d072d97a-b151-4c30-8ed5-ffe48b50817d\") " pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:31.960323 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:31.960282    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:32.092595 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:32.092563    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:32.094156 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:07:32.094130    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd072d97a_b151_4c30_8ed5_ffe48b50817d.slice/crio-e4bd2bfb44381158198bad954e54b6eefbc1a276cf0af9e0a734ab97442b437e WatchSource:0}: Error finding container e4bd2bfb44381158198bad954e54b6eefbc1a276cf0af9e0a734ab97442b437e: Status 404 returned error can't find the container with id e4bd2bfb44381158198bad954e54b6eefbc1a276cf0af9e0a734ab97442b437e
Jul 09 17:07:32.347490 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:32.347444    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238695984" event={"ID":"d072d97a-b151-4c30-8ed5-ffe48b50817d","Type":"ContainerStarted","Data":"e4bd2bfb44381158198bad954e54b6eefbc1a276cf0af9e0a734ab97442b437e"}
Jul 09 17:07:32.348745 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:32.348724    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb8a998a7dc8296d08409d3ac7d7a71da071fbd062ac46d9cfc60a0e068b34fc"
Jul 09 17:07:32.348888 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:32.348789    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:32.359839 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:32.359799    2586 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7234-140209238695984" oldPodUID="5326316d-c956-4a0f-b0f8-039be01d6509" podUID="d072d97a-b151-4c30-8ed5-ffe48b50817d"
Jul 09 17:07:32.773275 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:32.773172    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5326316d-c956-4a0f-b0f8-039be01d6509" path="/var/lib/kubelet/pods/5326316d-c956-4a0f-b0f8-039be01d6509/volumes"
Jul 09 17:07:33.356414 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:33.356381    2586 generic.go:356] "Generic (PLEG): container finished" podID="d072d97a-b151-4c30-8ed5-ffe48b50817d" containerID="136cb9f4813d5bce6c3befc6c376a0b942e62a3087eb34f03ca1a361fe0aedd7" exitCode=0
Jul 09 17:07:33.356589 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:33.356458    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238695984" event={"ID":"d072d97a-b151-4c30-8ed5-ffe48b50817d","Type":"ContainerDied","Data":"136cb9f4813d5bce6c3befc6c376a0b942e62a3087eb34f03ca1a361fe0aedd7"}
Jul 09 17:07:34.501646 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.501614    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:34.521984 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.521958    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7234-140209238695984_d072d97a-b151-4c30-8ed5-ffe48b50817d/test-curl-7234-140209238695984/0.log"
Jul 09 17:07:34.546645 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.546612    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:34.550703 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.550678    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7234-140209238695984"]
Jul 09 17:07:34.640613 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.640523    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mslp2\" (UniqueName: \"kubernetes.io/projected/d072d97a-b151-4c30-8ed5-ffe48b50817d-kube-api-access-mslp2\") pod \"d072d97a-b151-4c30-8ed5-ffe48b50817d\" (UID: \"d072d97a-b151-4c30-8ed5-ffe48b50817d\") "
Jul 09 17:07:34.642621 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.642598    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d072d97a-b151-4c30-8ed5-ffe48b50817d-kube-api-access-mslp2" (OuterVolumeSpecName: "kube-api-access-mslp2") pod "d072d97a-b151-4c30-8ed5-ffe48b50817d" (UID: "d072d97a-b151-4c30-8ed5-ffe48b50817d"). InnerVolumeSpecName "kube-api-access-mslp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:34.672044 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.672018    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:34.672382 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.672371    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d072d97a-b151-4c30-8ed5-ffe48b50817d" containerName="test-curl-7234-140209238695984"
Jul 09 17:07:34.672428 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.672384    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="d072d97a-b151-4c30-8ed5-ffe48b50817d" containerName="test-curl-7234-140209238695984"
Jul 09 17:07:34.672461 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.672445    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="d072d97a-b151-4c30-8ed5-ffe48b50817d" containerName="test-curl-7234-140209238695984"
Jul 09 17:07:34.675455 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.675439    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:34.679907 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.679874    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:34.741266 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.741232    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mslp2\" (UniqueName: \"kubernetes.io/projected/d072d97a-b151-4c30-8ed5-ffe48b50817d-kube-api-access-mslp2\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:34.773506 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.773471    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d072d97a-b151-4c30-8ed5-ffe48b50817d" path="/var/lib/kubelet/pods/d072d97a-b151-4c30-8ed5-ffe48b50817d/volumes"
Jul 09 17:07:34.842657 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.842613    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrcs\" (UniqueName: \"kubernetes.io/projected/35d3da95-ceee-4b81-90ab-eeedf4323a6e-kube-api-access-lsrcs\") pod \"test-curl-7234-140209238696848\" (UID: \"35d3da95-ceee-4b81-90ab-eeedf4323a6e\") " pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:34.943609 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.943511    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrcs\" (UniqueName: \"kubernetes.io/projected/35d3da95-ceee-4b81-90ab-eeedf4323a6e-kube-api-access-lsrcs\") pod \"test-curl-7234-140209238696848\" (UID: \"35d3da95-ceee-4b81-90ab-eeedf4323a6e\") " pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:34.951670 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.951637    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrcs\" (UniqueName: \"kubernetes.io/projected/35d3da95-ceee-4b81-90ab-eeedf4323a6e-kube-api-access-lsrcs\") pod \"test-curl-7234-140209238696848\" (UID: \"35d3da95-ceee-4b81-90ab-eeedf4323a6e\") " pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:34.985600 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:34.985572    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:35.117866 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:35.117693    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:35.120039 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:07:35.120010    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35d3da95_ceee_4b81_90ab_eeedf4323a6e.slice/crio-8d2705271e1c4d19bbafcba241ca45db38d2b9e4d7993c0a6c2bd3d4c75d6909 WatchSource:0}: Error finding container 8d2705271e1c4d19bbafcba241ca45db38d2b9e4d7993c0a6c2bd3d4c75d6909: Status 404 returned error can't find the container with id 8d2705271e1c4d19bbafcba241ca45db38d2b9e4d7993c0a6c2bd3d4c75d6909
Jul 09 17:07:35.367219 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:35.367187    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238695984"
Jul 09 17:07:35.367219 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:35.367214    2586 scope.go:117] "RemoveContainer" containerID="136cb9f4813d5bce6c3befc6c376a0b942e62a3087eb34f03ca1a361fe0aedd7"
Jul 09 17:07:35.368718 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:35.368681    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238696848" event={"ID":"35d3da95-ceee-4b81-90ab-eeedf4323a6e","Type":"ContainerStarted","Data":"8d2705271e1c4d19bbafcba241ca45db38d2b9e4d7993c0a6c2bd3d4c75d6909"}
Jul 09 17:07:36.376097 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:36.376061    2586 generic.go:356] "Generic (PLEG): container finished" podID="35d3da95-ceee-4b81-90ab-eeedf4323a6e" containerID="72482c8b328d3079a18c09de739f39d7e0f6d1415eed826247599b69676e6c3f" exitCode=0
Jul 09 17:07:36.376497 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:36.376154    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238696848" event={"ID":"35d3da95-ceee-4b81-90ab-eeedf4323a6e","Type":"ContainerDied","Data":"72482c8b328d3079a18c09de739f39d7e0f6d1415eed826247599b69676e6c3f"}
Jul 09 17:07:37.525029 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.524998    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:37.544445 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.544417    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7234-140209238696848_35d3da95-ceee-4b81-90ab-eeedf4323a6e/test-curl-7234-140209238696848/0.log"
Jul 09 17:07:37.569819 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.569784    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:37.572709 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.572687    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:37.665517 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.665430    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsrcs\" (UniqueName: \"kubernetes.io/projected/35d3da95-ceee-4b81-90ab-eeedf4323a6e-kube-api-access-lsrcs\") pod \"35d3da95-ceee-4b81-90ab-eeedf4323a6e\" (UID: \"35d3da95-ceee-4b81-90ab-eeedf4323a6e\") "
Jul 09 17:07:37.667576 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.667552    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35d3da95-ceee-4b81-90ab-eeedf4323a6e-kube-api-access-lsrcs" (OuterVolumeSpecName: "kube-api-access-lsrcs") pod "35d3da95-ceee-4b81-90ab-eeedf4323a6e" (UID: "35d3da95-ceee-4b81-90ab-eeedf4323a6e"). InnerVolumeSpecName "kube-api-access-lsrcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:37.695592 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.695558    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:37.695953 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.695941    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35d3da95-ceee-4b81-90ab-eeedf4323a6e" containerName="test-curl-7234-140209238696848"
Jul 09 17:07:37.696012 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.695955    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="35d3da95-ceee-4b81-90ab-eeedf4323a6e" containerName="test-curl-7234-140209238696848"
Jul 09 17:07:37.696050 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.696028    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="35d3da95-ceee-4b81-90ab-eeedf4323a6e" containerName="test-curl-7234-140209238696848"
Jul 09 17:07:37.698989 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.698973    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:37.705557 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.705522    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:37.766433 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.766393    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsrcs\" (UniqueName: \"kubernetes.io/projected/35d3da95-ceee-4b81-90ab-eeedf4323a6e-kube-api-access-lsrcs\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:37.867777 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.867731    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvtgt\" (UniqueName: \"kubernetes.io/projected/15cf28bd-9aad-4bfc-8619-707b5aaa516d-kube-api-access-pvtgt\") pod \"test-curl-7234-140209238696848\" (UID: \"15cf28bd-9aad-4bfc-8619-707b5aaa516d\") " pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:37.968934 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.968840    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvtgt\" (UniqueName: \"kubernetes.io/projected/15cf28bd-9aad-4bfc-8619-707b5aaa516d-kube-api-access-pvtgt\") pod \"test-curl-7234-140209238696848\" (UID: \"15cf28bd-9aad-4bfc-8619-707b5aaa516d\") " pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:37.976755 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:37.976722    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvtgt\" (UniqueName: \"kubernetes.io/projected/15cf28bd-9aad-4bfc-8619-707b5aaa516d-kube-api-access-pvtgt\") pod \"test-curl-7234-140209238696848\" (UID: \"15cf28bd-9aad-4bfc-8619-707b5aaa516d\") " pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:38.009488 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.009452    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:38.138256 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.138232    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:38.139566 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:07:38.139533    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15cf28bd_9aad_4bfc_8619_707b5aaa516d.slice/crio-ef94319cf361db2cfb1d2157bc60238c3473bd878a91d9629ef4f20275b482ff WatchSource:0}: Error finding container ef94319cf361db2cfb1d2157bc60238c3473bd878a91d9629ef4f20275b482ff: Status 404 returned error can't find the container with id ef94319cf361db2cfb1d2157bc60238c3473bd878a91d9629ef4f20275b482ff
Jul 09 17:07:38.388653 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.388617    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238696848" event={"ID":"15cf28bd-9aad-4bfc-8619-707b5aaa516d","Type":"ContainerStarted","Data":"ef94319cf361db2cfb1d2157bc60238c3473bd878a91d9629ef4f20275b482ff"}
Jul 09 17:07:38.389779 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.389731    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:38.389779 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.389775    2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d2705271e1c4d19bbafcba241ca45db38d2b9e4d7993c0a6c2bd3d4c75d6909"
Jul 09 17:07:38.400541 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.400511    2586 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7234-140209238696848" oldPodUID="35d3da95-ceee-4b81-90ab-eeedf4323a6e" podUID="15cf28bd-9aad-4bfc-8619-707b5aaa516d"
Jul 09 17:07:38.772640 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:38.772559    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35d3da95-ceee-4b81-90ab-eeedf4323a6e" path="/var/lib/kubelet/pods/35d3da95-ceee-4b81-90ab-eeedf4323a6e/volumes"
Jul 09 17:07:39.397108 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:39.397077    2586 generic.go:356] "Generic (PLEG): container finished" podID="15cf28bd-9aad-4bfc-8619-707b5aaa516d" containerID="d659463ba503293555e3284449af591f23eb2e86f2250d101ec9f9e59d7929f5" exitCode=0
Jul 09 17:07:39.397283 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:39.397140    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7234-140209238696848" event={"ID":"15cf28bd-9aad-4bfc-8619-707b5aaa516d","Type":"ContainerDied","Data":"d659463ba503293555e3284449af591f23eb2e86f2250d101ec9f9e59d7929f5"}
Jul 09 17:07:40.543209 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.543183    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:40.563058 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.563032    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7234-140209238696848_15cf28bd-9aad-4bfc-8619-707b5aaa516d/test-curl-7234-140209238696848/0.log"
Jul 09 17:07:40.587791 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.587728    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:40.589692 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.589671    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7234-140209238696848"]
Jul 09 17:07:40.695992 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.695890    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvtgt\" (UniqueName: \"kubernetes.io/projected/15cf28bd-9aad-4bfc-8619-707b5aaa516d-kube-api-access-pvtgt\") pod \"15cf28bd-9aad-4bfc-8619-707b5aaa516d\" (UID: \"15cf28bd-9aad-4bfc-8619-707b5aaa516d\") "
Jul 09 17:07:40.697983 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.697952    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15cf28bd-9aad-4bfc-8619-707b5aaa516d-kube-api-access-pvtgt" (OuterVolumeSpecName: "kube-api-access-pvtgt") pod "15cf28bd-9aad-4bfc-8619-707b5aaa516d" (UID: "15cf28bd-9aad-4bfc-8619-707b5aaa516d"). InnerVolumeSpecName "kube-api-access-pvtgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:40.773654 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.773615    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15cf28bd-9aad-4bfc-8619-707b5aaa516d" path="/var/lib/kubelet/pods/15cf28bd-9aad-4bfc-8619-707b5aaa516d/volumes"
Jul 09 17:07:40.796844 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:40.796811    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvtgt\" (UniqueName: \"kubernetes.io/projected/15cf28bd-9aad-4bfc-8619-707b5aaa516d-kube-api-access-pvtgt\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:41.409464 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:41.409427    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7234-140209238696848"
Jul 09 17:07:41.409628 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:41.409472    2586 scope.go:117] "RemoveContainer" containerID="d659463ba503293555e3284449af591f23eb2e86f2250d101ec9f9e59d7929f5"
Jul 09 17:07:41.507681 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:41.507648    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"]
Jul 09 17:07:41.507965 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:41.507908    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" podUID="b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" containerName="maas-api" containerID="cri-o://da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700" gracePeriod=30
Jul 09 17:07:41.970645 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:41.970619    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:07:42.017872 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.017836    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfgk9\" (UniqueName: \"kubernetes.io/projected/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-kube-api-access-vfgk9\") pod \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") "
Jul 09 17:07:42.017872 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.017886    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls\") pod \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") "
Jul 09 17:07:42.018132 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.017936    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-openshift-service-ca\") pod \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\" (UID: \"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6\") "
Jul 09 17:07:42.018283 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.018255    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" (UID: "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 17:07:42.019992 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.019958    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-kube-api-access-vfgk9" (OuterVolumeSpecName: "kube-api-access-vfgk9") pod "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" (UID: "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6"). InnerVolumeSpecName "kube-api-access-vfgk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:42.019992 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.019974    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" (UID: "b9d20cb4-1405-4d6e-9c1d-8026712fd7b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:07:42.119322 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.119283    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfgk9\" (UniqueName: \"kubernetes.io/projected/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-kube-api-access-vfgk9\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:42.119322 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.119313    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-maas-api-tls\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:42.119322 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.119322    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6-openshift-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:42.418111 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.418007    2586 generic.go:356] "Generic (PLEG): container finished" podID="b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" containerID="da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700" exitCode=0
Jul 09 17:07:42.418111 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.418052    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"
Jul 09 17:07:42.418111 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.418090    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" event={"ID":"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6","Type":"ContainerDied","Data":"da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700"}
Jul 09 17:07:42.418337 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.418145    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8" event={"ID":"b9d20cb4-1405-4d6e-9c1d-8026712fd7b6","Type":"ContainerDied","Data":"f0c4122729268d84015bf7a723debacccb153d8ea40247bdebd1db4e728479c3"}
Jul 09 17:07:42.418337 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.418167    2586 scope.go:117] "RemoveContainer" containerID="da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700"
Jul 09 17:07:42.427839 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.427819    2586 scope.go:117] "RemoveContainer" containerID="da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700"
Jul 09 17:07:42.428124 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:07:42.428107    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700\": container with ID starting with da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700 not found: ID does not exist" containerID="da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700"
Jul 09 17:07:42.428172 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.428134    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700"} err="failed to get container status \"da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700\": rpc error: code = NotFound desc = could not find container \"da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700\": container with ID starting with da67119910da37bef57700fdb84f9c57cc3a74d3233563fd74961f6b6bd40700 not found: ID does not exist"
Jul 09 17:07:42.438257 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.438223    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"]
Jul 09 17:07:42.440931 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.440908    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-d1f7bd-66b965b6cb-nkpz8"]
Jul 09 17:07:42.772733 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:42.772651    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" path="/var/lib/kubelet/pods/b9d20cb4-1405-4d6e-9c1d-8026712fd7b6/volumes"
Jul 09 17:07:44.729108 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:44.729070    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:07:48.737422 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:48.737390    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"]
Jul 09 17:07:48.737958 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:48.737599    2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" podUID="1574c334-3609-4d1d-9a24-c3b67e491744" containerName="maas-api" containerID="cri-o://5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9" gracePeriod=30
Jul 09 17:07:48.989117 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:48.989057    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:07:49.075431 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.075391    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bm5zx\" (UniqueName: \"kubernetes.io/projected/1574c334-3609-4d1d-9a24-c3b67e491744-kube-api-access-bm5zx\") pod \"1574c334-3609-4d1d-9a24-c3b67e491744\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") "
Jul 09 17:07:49.075581 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.075441    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1574c334-3609-4d1d-9a24-c3b67e491744-openshift-service-ca\") pod \"1574c334-3609-4d1d-9a24-c3b67e491744\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") "
Jul 09 17:07:49.075581 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.075477    2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls\") pod \"1574c334-3609-4d1d-9a24-c3b67e491744\" (UID: \"1574c334-3609-4d1d-9a24-c3b67e491744\") "
Jul 09 17:07:49.075884 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.075857    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1574c334-3609-4d1d-9a24-c3b67e491744-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1574c334-3609-4d1d-9a24-c3b67e491744" (UID: "1574c334-3609-4d1d-9a24-c3b67e491744"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 17:07:49.077630 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.077598    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1574c334-3609-4d1d-9a24-c3b67e491744" (UID: "1574c334-3609-4d1d-9a24-c3b67e491744"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 17:07:49.077721 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.077654    2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1574c334-3609-4d1d-9a24-c3b67e491744-kube-api-access-bm5zx" (OuterVolumeSpecName: "kube-api-access-bm5zx") pod "1574c334-3609-4d1d-9a24-c3b67e491744" (UID: "1574c334-3609-4d1d-9a24-c3b67e491744"). InnerVolumeSpecName "kube-api-access-bm5zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 17:07:49.176312 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.176278    2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm5zx\" (UniqueName: \"kubernetes.io/projected/1574c334-3609-4d1d-9a24-c3b67e491744-kube-api-access-bm5zx\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:49.176312 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.176309    2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1574c334-3609-4d1d-9a24-c3b67e491744-openshift-service-ca\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:49.176564 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.176325    2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1574c334-3609-4d1d-9a24-c3b67e491744-maas-api-tls\") on node \"ip-10-0-139-176.ec2.internal\" DevicePath \"\""
Jul 09 17:07:49.457670 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.457638    2586 generic.go:356] "Generic (PLEG): container finished" podID="1574c334-3609-4d1d-9a24-c3b67e491744" containerID="5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9" exitCode=0
Jul 09 17:07:49.457886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.457680    2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"
Jul 09 17:07:49.457886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.457723    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" event={"ID":"1574c334-3609-4d1d-9a24-c3b67e491744","Type":"ContainerDied","Data":"5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9"}
Jul 09 17:07:49.457886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.457800    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh" event={"ID":"1574c334-3609-4d1d-9a24-c3b67e491744","Type":"ContainerDied","Data":"30b96700074e4d6c2cd8dd8b312e603b96d3f4cb57ba022ddaa439356f5e4265"}
Jul 09 17:07:49.457886 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.457830    2586 scope.go:117] "RemoveContainer" containerID="5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9"
Jul 09 17:07:49.467647 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.467624    2586 scope.go:117] "RemoveContainer" containerID="5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9"
Jul 09 17:07:49.467959 ip-10-0-139-176 kubenswrapper[2586]: E0709 17:07:49.467941    2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9\": container with ID starting with 5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9 not found: ID does not exist" containerID="5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9"
Jul 09 17:07:49.468018 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.467968    2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9"} err="failed to get container status \"5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9\": rpc error: code = NotFound desc = could not find container \"5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9\": container with ID starting with 5d978d8debd0808866847c162e2d1bfa3f2af14c250a0396f75d5f529e030fc9 not found: ID does not exist"
Jul 09 17:07:49.479852 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.479821    2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"]
Jul 09 17:07:49.483845 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:49.483817    2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e779af-54565c4c7d-26qhh"]
Jul 09 17:07:50.773956 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:50.773921    2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1574c334-3609-4d1d-9a24-c3b67e491744" path="/var/lib/kubelet/pods/1574c334-3609-4d1d-9a24-c3b67e491744/volumes"
Jul 09 17:07:55.083784 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:55.083725    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-ffd45bb66-4spxv_b3aae518-6657-413e-8c5e-cd9719e6bb7a/authorino/0.log"
Jul 09 17:07:57.324786 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:07:57.324725    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cplnw"]
Jul 09 17:08:00.278772 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:00.278726    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9nz6j_7895c73c-a4ad-41f2-be7b-969dfec89911/server/0.log"
Jul 09 17:08:00.401642 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:00.401609    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kbwqh_fa630290-6a31-4db5-a88d-d87afa2b19f4/manager/0.log"
Jul 09 17:08:00.648896 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:00.648864    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bd696597f-k5htn_fb6af1b2-ed62-4334-a45f-2074c48228d2/copy-manifests/0.log"
Jul 09 17:08:00.655525 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:00.655497    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bd696597f-k5htn_fb6af1b2-ed62-4334-a45f-2074c48228d2/manager/0.log"
Jul 09 17:08:02.017487 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.017457    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr_4fd28db9-ca9f-47bc-b417-b822a2a653af/util/0.log"
Jul 09 17:08:02.024382 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.024345    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr_4fd28db9-ca9f-47bc-b417-b822a2a653af/pull/0.log"
Jul 09 17:08:02.031884 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.031859    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr_4fd28db9-ca9f-47bc-b417-b822a2a653af/extract/0.log"
Jul 09 17:08:02.142934 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.142906    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6_8e65f840-c5f3-4ee5-9901-db6192eb8577/util/0.log"
Jul 09 17:08:02.149370 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.149342    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6_8e65f840-c5f3-4ee5-9901-db6192eb8577/pull/0.log"
Jul 09 17:08:02.155777 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.155743    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6_8e65f840-c5f3-4ee5-9901-db6192eb8577/extract/0.log"
Jul 09 17:08:02.278482 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.278396    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc_ad61d77a-3a6f-4545-b92d-6e86bf8176e0/util/0.log"
Jul 09 17:08:02.285196 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.285173    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc_ad61d77a-3a6f-4545-b92d-6e86bf8176e0/pull/0.log"
Jul 09 17:08:02.291277 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.291252    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc_ad61d77a-3a6f-4545-b92d-6e86bf8176e0/extract/0.log"
Jul 09 17:08:02.403152 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.403122    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz_1915b720-a446-4114-b498-d96f1292a493/util/0.log"
Jul 09 17:08:02.410086 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.410052    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz_1915b720-a446-4114-b498-d96f1292a493/pull/0.log"
Jul 09 17:08:02.416688 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.416663    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz_1915b720-a446-4114-b498-d96f1292a493/extract/0.log"
Jul 09 17:08:02.539567 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.539486    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-ffd45bb66-4spxv_b3aae518-6657-413e-8c5e-cd9719e6bb7a/authorino/0.log"
Jul 09 17:08:02.873123 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:02.873096    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-lgqqs_662e7b88-5a52-40ff-8cd9-7ee9acb9f830/kuadrant-console-plugin/0.log"
Jul 09 17:08:03.218559 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:03.218470    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cplnw_8f86287e-03ef-4445-a435-4c97ce49b560/limitador/0.log"
Jul 09 17:08:04.026519 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:04.026485    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-868f9f885b-tpg7f_eaa6961b-be84-4761-b668-ddd030e2c902/kube-auth-proxy/0.log"
Jul 09 17:08:05.388059 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:05.388028    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-qnlk5_8f361d48-a40f-4706-90a8-13fca4e7b8b2/main/0.log"
Jul 09 17:08:09.755702 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.755661    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7jqtn/must-gather-tjtxp"]
Jul 09 17:08:09.756531 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.756507    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1574c334-3609-4d1d-9a24-c3b67e491744" containerName="maas-api"
Jul 09 17:08:09.756702 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.756689    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="1574c334-3609-4d1d-9a24-c3b67e491744" containerName="maas-api"
Jul 09 17:08:09.756834 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.756816    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" containerName="maas-api"
Jul 09 17:08:09.756834 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.756831    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" containerName="maas-api"
Jul 09 17:08:09.757019 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.756849    2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15cf28bd-9aad-4bfc-8619-707b5aaa516d" containerName="test-curl-7234-140209238696848"
Jul 09 17:08:09.757019 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.756858    2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="15cf28bd-9aad-4bfc-8619-707b5aaa516d" containerName="test-curl-7234-140209238696848"
Jul 09 17:08:09.757123 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.757056    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="15cf28bd-9aad-4bfc-8619-707b5aaa516d" containerName="test-curl-7234-140209238696848"
Jul 09 17:08:09.757123 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.757068    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9d20cb4-1405-4d6e-9c1d-8026712fd7b6" containerName="maas-api"
Jul 09 17:08:09.757123 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.757084    2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="1574c334-3609-4d1d-9a24-c3b67e491744" containerName="maas-api"
Jul 09 17:08:09.761268 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.761245    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:09.765053 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.764993    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7jqtn\"/\"kube-root-ca.crt\""
Jul 09 17:08:09.765993 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.765967    2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7jqtn\"/\"openshift-service-ca.crt\""
Jul 09 17:08:09.768181 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.768157    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jqtn/must-gather-tjtxp"]
Jul 09 17:08:09.857048 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.856998    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9946130b-82f7-4e61-9746-d8d849662d2a-must-gather-output\") pod \"must-gather-tjtxp\" (UID: \"9946130b-82f7-4e61-9746-d8d849662d2a\") " pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:09.857255 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.857105    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4qcd\" (UniqueName: \"kubernetes.io/projected/9946130b-82f7-4e61-9746-d8d849662d2a-kube-api-access-l4qcd\") pod \"must-gather-tjtxp\" (UID: \"9946130b-82f7-4e61-9746-d8d849662d2a\") " pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:09.958082 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.958046    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4qcd\" (UniqueName: \"kubernetes.io/projected/9946130b-82f7-4e61-9746-d8d849662d2a-kube-api-access-l4qcd\") pod \"must-gather-tjtxp\" (UID: \"9946130b-82f7-4e61-9746-d8d849662d2a\") " pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:09.958273 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.958163    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9946130b-82f7-4e61-9746-d8d849662d2a-must-gather-output\") pod \"must-gather-tjtxp\" (UID: \"9946130b-82f7-4e61-9746-d8d849662d2a\") " pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:09.958490 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.958470    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9946130b-82f7-4e61-9746-d8d849662d2a-must-gather-output\") pod \"must-gather-tjtxp\" (UID: \"9946130b-82f7-4e61-9746-d8d849662d2a\") " pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:09.969117 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:09.969088    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4qcd\" (UniqueName: \"kubernetes.io/projected/9946130b-82f7-4e61-9746-d8d849662d2a-kube-api-access-l4qcd\") pod \"must-gather-tjtxp\" (UID: \"9946130b-82f7-4e61-9746-d8d849662d2a\") " pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:10.071690 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:10.071598    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jqtn/must-gather-tjtxp"
Jul 09 17:08:10.221650 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:10.221619    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jqtn/must-gather-tjtxp"]
Jul 09 17:08:10.223164 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:08:10.223128    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9946130b_82f7_4e61_9746_d8d849662d2a.slice/crio-d080f12e607f4e015c3f57df7ddb23a7afafe14b940853897a8d1ede56328b76 WatchSource:0}: Error finding container d080f12e607f4e015c3f57df7ddb23a7afafe14b940853897a8d1ede56328b76: Status 404 returned error can't find the container with id d080f12e607f4e015c3f57df7ddb23a7afafe14b940853897a8d1ede56328b76
Jul 09 17:08:10.575267 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:10.575236    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jqtn/must-gather-tjtxp" event={"ID":"9946130b-82f7-4e61-9746-d8d849662d2a","Type":"ContainerStarted","Data":"d080f12e607f4e015c3f57df7ddb23a7afafe14b940853897a8d1ede56328b76"}
Jul 09 17:08:11.583539 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:11.583502    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jqtn/must-gather-tjtxp" event={"ID":"9946130b-82f7-4e61-9746-d8d849662d2a","Type":"ContainerStarted","Data":"049ca2d938baf85792c5f629ce0d97299a3275536301e78067fc66d7d59c9e9e"}
Jul 09 17:08:11.583539 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:11.583552    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jqtn/must-gather-tjtxp" event={"ID":"9946130b-82f7-4e61-9746-d8d849662d2a","Type":"ContainerStarted","Data":"1724fb03fdc7dd322b64c7635084a283a8301c904dbf86e12cd7250645389efd"}
Jul 09 17:08:11.603426 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:11.603356    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7jqtn/must-gather-tjtxp" podStartSLOduration=1.851485699 podStartE2EDuration="2.603334747s" totalImagesPullingTime="751.849048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:08:11.599267711 +0000 UTC m=+2902.525221991" watchObservedRunningTime="2026-07-09 17:08:11.603334747 +0000 UTC m=+2902.529289026"
Jul 09 17:08:12.511652 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:12.511623    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fn7v8_ba0bf9e7-94ef-430a-915a-f80caa23e641/global-pull-secret-syncer/0.log"
Jul 09 17:08:12.716386 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:12.716349    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mqdxc_d34ad955-4c65-4b5c-ac9c-818e4b3a12fd/konnectivity-agent/0.log"
Jul 09 17:08:12.790872 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:12.790790    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-176.ec2.internal_05ff5927c1fde4b3df7bf6c1bc3393e1/haproxy/0.log"
Jul 09 17:08:17.556811 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.556778    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr_4fd28db9-ca9f-47bc-b417-b822a2a653af/extract/0.log"
Jul 09 17:08:17.579861 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.579707    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr_4fd28db9-ca9f-47bc-b417-b822a2a653af/util/0.log"
Jul 09 17:08:17.603161 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.603135    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759n6jxr_4fd28db9-ca9f-47bc-b417-b822a2a653af/pull/0.log"
Jul 09 17:08:17.635669 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.635640    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6_8e65f840-c5f3-4ee5-9901-db6192eb8577/extract/0.log"
Jul 09 17:08:17.662814 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.662788    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6_8e65f840-c5f3-4ee5-9901-db6192eb8577/util/0.log"
Jul 09 17:08:17.684281 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.684249    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0sbbh6_8e65f840-c5f3-4ee5-9901-db6192eb8577/pull/0.log"
Jul 09 17:08:17.710956 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.710925    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc_ad61d77a-3a6f-4545-b92d-6e86bf8176e0/extract/0.log"
Jul 09 17:08:17.740895 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.740865    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc_ad61d77a-3a6f-4545-b92d-6e86bf8176e0/util/0.log"
Jul 09 17:08:17.765292 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.765264    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p6tbc_ad61d77a-3a6f-4545-b92d-6e86bf8176e0/pull/0.log"
Jul 09 17:08:17.808321 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.808241    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz_1915b720-a446-4114-b498-d96f1292a493/extract/0.log"
Jul 09 17:08:17.830720 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.830696    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz_1915b720-a446-4114-b498-d96f1292a493/util/0.log"
Jul 09 17:08:17.855800 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:17.855750    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gqccz_1915b720-a446-4114-b498-d96f1292a493/pull/0.log"
Jul 09 17:08:18.358989 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:18.358954    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-ffd45bb66-4spxv_b3aae518-6657-413e-8c5e-cd9719e6bb7a/authorino/0.log"
Jul 09 17:08:18.457686 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:18.457663    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-lgqqs_662e7b88-5a52-40ff-8cd9-7ee9acb9f830/kuadrant-console-plugin/0.log"
Jul 09 17:08:18.730103 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:18.730014    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cplnw_8f86287e-03ef-4445-a435-4c97ce49b560/limitador/0.log"
Jul 09 17:08:20.428995 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.428960    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/alertmanager/0.log"
Jul 09 17:08:20.454303 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.454265    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/config-reloader/0.log"
Jul 09 17:08:20.476783 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.476727    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/kube-rbac-proxy-web/0.log"
Jul 09 17:08:20.502653 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.502619    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/kube-rbac-proxy/0.log"
Jul 09 17:08:20.526438 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.526400    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/kube-rbac-proxy-metric/0.log"
Jul 09 17:08:20.555103 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.555062    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/prom-label-proxy/0.log"
Jul 09 17:08:20.579263 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.579202    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6989644a-4bf7-4977-8978-274e296c0db0/init-config-reloader/0.log"
Jul 09 17:08:20.790324 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.790224    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-d2rcf_01315eef-d12c-4064-9b36-b1e5426a6086/monitoring-plugin/0.log"
Jul 09 17:08:20.912011 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.911959    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lqcnq_7104417e-1224-4939-bca1-4a18b5ddc97e/node-exporter/0.log"
Jul 09 17:08:20.933044 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.933016    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lqcnq_7104417e-1224-4939-bca1-4a18b5ddc97e/kube-rbac-proxy/0.log"
Jul 09 17:08:20.955008 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:20.954979    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lqcnq_7104417e-1224-4939-bca1-4a18b5ddc97e/init-textfile/0.log"
Jul 09 17:08:21.067843 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.067743    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-n9mz7_78a3efd9-412b-4cdb-bdf8-a4a1d965b129/kube-rbac-proxy-main/0.log"
Jul 09 17:08:21.091003 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.090974    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-n9mz7_78a3efd9-412b-4cdb-bdf8-a4a1d965b129/kube-rbac-proxy-self/0.log"
Jul 09 17:08:21.113839 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.113804    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-n9mz7_78a3efd9-412b-4cdb-bdf8-a4a1d965b129/openshift-state-metrics/0.log"
Jul 09 17:08:21.339951 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.339916    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-fkn6x_cfed5735-fff9-4abd-b879-c20b6ac9bc14/prometheus-operator/0.log"
Jul 09 17:08:21.361078 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.361048    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-fkn6x_cfed5735-fff9-4abd-b879-c20b6ac9bc14/kube-rbac-proxy/0.log"
Jul 09 17:08:21.529728 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.529699    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b9cd557-z8tvs_8120edc8-a045-4f1c-84fc-a18706e65419/thanos-query/0.log"
Jul 09 17:08:21.559316 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.559294    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b9cd557-z8tvs_8120edc8-a045-4f1c-84fc-a18706e65419/kube-rbac-proxy-web/0.log"
Jul 09 17:08:21.592468 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.592387    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b9cd557-z8tvs_8120edc8-a045-4f1c-84fc-a18706e65419/kube-rbac-proxy/0.log"
Jul 09 17:08:21.623523 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.623497    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b9cd557-z8tvs_8120edc8-a045-4f1c-84fc-a18706e65419/prom-label-proxy/0.log"
Jul 09 17:08:21.648644 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.648613    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b9cd557-z8tvs_8120edc8-a045-4f1c-84fc-a18706e65419/kube-rbac-proxy-rules/0.log"
Jul 09 17:08:21.672238 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:21.672204    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-866b9cd557-z8tvs_8120edc8-a045-4f1c-84fc-a18706e65419/kube-rbac-proxy-metrics/0.log"
Jul 09 17:08:23.398528 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.398494    2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"]
Jul 09 17:08:23.404839 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.404808    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.407811 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.407655    2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7jqtn\"/\"default-dockercfg-r67xr\""
Jul 09 17:08:23.414137 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.412424    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"]
Jul 09 17:08:23.500163 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.500128    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-proc\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.500365 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.500242    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw78f\" (UniqueName: \"kubernetes.io/projected/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-kube-api-access-fw78f\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.500365 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.500279    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-sys\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.500365 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.500307    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-podres\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.500365 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.500331    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-lib-modules\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601230 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601190    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw78f\" (UniqueName: \"kubernetes.io/projected/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-kube-api-access-fw78f\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601429 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601242    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-sys\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601429 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601272    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-podres\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601429 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601329    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-lib-modules\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601429 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601414    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-podres\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601429 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601418    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-sys\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601659 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601432    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-proc\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601659 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601517    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-lib-modules\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.601659 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.601555    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-proc\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.609646 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.609610    2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw78f\" (UniqueName: \"kubernetes.io/projected/5d258ade-d12a-4a3e-8cec-4dbd6b69383b-kube-api-access-fw78f\") pod \"perf-node-gather-daemonset-599zh\" (UID: \"5d258ade-d12a-4a3e-8cec-4dbd6b69383b\") " pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.724318 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.724230    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:23.898633 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:23.898582    2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"]
Jul 09 17:08:23.901080 ip-10-0-139-176 kubenswrapper[2586]: W0709 17:08:23.901048    2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5d258ade_d12a_4a3e_8cec_4dbd6b69383b.slice/crio-729fbe1274be6fdb5c806ad45f8d371ac66d1cb137cfcc8779ba25ba94aed307 WatchSource:0}: Error finding container 729fbe1274be6fdb5c806ad45f8d371ac66d1cb137cfcc8779ba25ba94aed307: Status 404 returned error can't find the container with id 729fbe1274be6fdb5c806ad45f8d371ac66d1cb137cfcc8779ba25ba94aed307
Jul 09 17:08:24.686749 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:24.686700    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh" event={"ID":"5d258ade-d12a-4a3e-8cec-4dbd6b69383b","Type":"ContainerStarted","Data":"0072f82cdd749b16ff7c6d6660e75126a7a28ae73b52d4c4bc8aa6ab0b637a87"}
Jul 09 17:08:24.687265 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:24.686922    2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh" event={"ID":"5d258ade-d12a-4a3e-8cec-4dbd6b69383b","Type":"ContainerStarted","Data":"729fbe1274be6fdb5c806ad45f8d371ac66d1cb137cfcc8779ba25ba94aed307"}
Jul 09 17:08:24.687265 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:24.687245    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:24.706151 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:24.706095    2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh" podStartSLOduration=1.706078464 podStartE2EDuration="1.706078464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:08:24.703958084 +0000 UTC m=+2915.629912363" watchObservedRunningTime="2026-07-09 17:08:24.706078464 +0000 UTC m=+2915.632032783"
Jul 09 17:08:25.483403 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:25.483371    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzgfq_3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb/dns/0.log"
Jul 09 17:08:25.505653 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:25.505620    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzgfq_3f4fa0a2-b1f8-4f17-b864-6ff5f4079fbb/kube-rbac-proxy/0.log"
Jul 09 17:08:25.655694 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:25.655663    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ms7gh_ffba687f-78d4-45a2-8110-1df0cdbc4b40/dns-node-resolver/0.log"
Jul 09 17:08:26.268445 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:26.268414    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rqpgb_4987e40b-3a77-46bc-9896-90246ddd8ce4/node-ca/0.log"
Jul 09 17:08:27.445219 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:27.445183    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-868f9f885b-tpg7f_eaa6961b-be84-4761-b668-ddd030e2c902/kube-auth-proxy/0.log"
Jul 09 17:08:28.433287 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:28.433257    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mgpzc_e68249db-71d5-47d9-b11a-d636f7967a37/serve-healthcheck-canary/0.log"
Jul 09 17:08:29.039068 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:29.039035    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9fbtj_10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e/kube-rbac-proxy/0.log"
Jul 09 17:08:29.061017 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:29.060984    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9fbtj_10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e/exporter/0.log"
Jul 09 17:08:29.084110 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:29.084083    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9fbtj_10d53b6a-70d9-4eeb-a4cf-9b51e1000e9e/extractor/0.log"
Jul 09 17:08:30.704840 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:30.704811    2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7jqtn/perf-node-gather-daemonset-599zh"
Jul 09 17:08:31.521605 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:31.521574    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9nz6j_7895c73c-a4ad-41f2-be7b-969dfec89911/server/0.log"
Jul 09 17:08:31.636272 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:31.636235    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kbwqh_fa630290-6a31-4db5-a88d-d87afa2b19f4/manager/0.log"
Jul 09 17:08:31.766645 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:31.766609    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bd696597f-k5htn_fb6af1b2-ed62-4334-a45f-2074c48228d2/manager/0.log"
Jul 09 17:08:31.788559 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:31.788469    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bd696597f-k5htn_fb6af1b2-ed62-4334-a45f-2074c48228d2/copy-manifests/0.log"
Jul 09 17:08:33.299016 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:33.298970    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7d5799cc79-pnzfj_55e78e6f-921d-46e6-a1a0-a6dae97c2afc/manager/0.log"
Jul 09 17:08:33.354212 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:33.354168    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-742mk_8fa40253-60f5-43aa-a724-ee6da7be1f43/openshift-lws-operator/0.log"
Jul 09 17:08:33.677695 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:33.677669    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-868f9f885b-tpg7f_eaa6961b-be84-4761-b668-ddd030e2c902/kube-auth-proxy/0.log"
Jul 09 17:08:35.127575 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:35.127541    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-868f9f885b-tpg7f_eaa6961b-be84-4761-b668-ddd030e2c902/kube-auth-proxy/0.log"
Jul 09 17:08:40.127574 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.127550    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/kube-multus-additional-cni-plugins/0.log"
Jul 09 17:08:40.158054 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.158024    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/egress-router-binary-copy/0.log"
Jul 09 17:08:40.189912 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.189879    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/cni-plugins/0.log"
Jul 09 17:08:40.222625 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.222597    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/bond-cni-plugin/0.log"
Jul 09 17:08:40.245410 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.245380    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/routeoverride-cni/0.log"
Jul 09 17:08:40.269523 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.269447    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/whereabouts-cni-bincopy/0.log"
Jul 09 17:08:40.295332 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.295299    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v5xwb_48466a63-02d5-4325-a3c1-6e561a9a01b1/whereabouts-cni/0.log"
Jul 09 17:08:40.345552 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.345521    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q25q9_4145b244-d40e-4bf6-b24c-fc0f31e6deef/kube-multus/0.log"
Jul 09 17:08:40.435116 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.435092    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5p8v6_8cbcc858-6c65-4abf-b5ca-62934d5952c3/network-metrics-daemon/0.log"
Jul 09 17:08:40.454081 ip-10-0-139-176 kubenswrapper[2586]: I0709 17:08:40.454054    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5p8v6_8cbcc858-6c65-4abf-b5ca-62934d5952c3/kube-rbac-proxy/0.log"