Jul 13 23:37:19.396462 ip-10-0-137-167 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 23:37:19.396472 ip-10-0-137-167 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 23:37:19.396479 ip-10-0-137-167 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 23:37:19.396725 ip-10-0-137-167 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 23:37:29.402717 ip-10-0-137-167 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 23:37:29.402733 ip-10-0-137-167 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b0d3f215868f4ab38adb4d5a71b89d59 --
Jul 13 23:39:06.983695 ip-10-0-137-167 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 23:39:07.428052 ip-10-0-137-167 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 23:39:07.428052 ip-10-0-137-167 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 23:39:07.428052 ip-10-0-137-167 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 23:39:07.428052 ip-10-0-137-167 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 23:39:07.428052 ip-10-0-137-167 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 23:39:07.428684 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.428103    2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 23:39:07.432188 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432173    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 23:39:07.432188 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432189    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432193    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432196    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432198    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432202    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432205    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432208    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432211    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432213    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432218    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432221    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432224    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432227    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432230    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432232    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432235    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432237    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432240    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432242    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 23:39:07.432256 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432245    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432247    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432250    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432252    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432255    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432258    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432260    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432263    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432266    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432270    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432273    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432276    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432278    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432281    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432283    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432286    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432288    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432291    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432293    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432296    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 23:39:07.432744 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432299    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432301    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432304    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432306    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432308    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432311    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432313    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432316    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432318    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432320    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432323    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432326    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432328    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432330    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432333    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432336    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432339    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432341    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432343    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432346    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 23:39:07.433282 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432348    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432351    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432353    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432356    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432358    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432360    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432363    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432365    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432367    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432369    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432372    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432375    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432377    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432380    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432382    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432384    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432387    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432390    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432393    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432395    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 23:39:07.439532 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432397    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432400    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432402    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432405    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432408    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432410    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432413    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432431    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432434    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432437    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432439    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432441    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432444    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432446    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432449    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432452    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432454    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432458    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.432461    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 23:39:07.440829 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433152    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433158    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433163    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433166    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433169    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433172    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433175    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433178    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433180    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433183    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433186    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433189    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433191    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433194    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433197    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433200    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433202    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433205    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433208    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433211    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 23:39:07.442673 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433213    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433216    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433218    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433221    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433224    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433226    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433229    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433231    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433234    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433236    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433239    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433241    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433244    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433246    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433248    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433251    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433253    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433255    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433259    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433261    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 23:39:07.443168 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433263    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433266    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433269    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433271    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433273    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433276    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433278    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433282    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433285    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433287    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433290    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433293    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433297    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433301    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433304    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433307    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433310    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433313    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433316    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433318    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 23:39:07.443674 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433321    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433323    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433325    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433328    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433331    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433333    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433335    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433338    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433340    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433342    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433345    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433348    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433350    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433353    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433355    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433358    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433360    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433363    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433365    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433367    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 23:39:07.444150 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433370    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433372    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433375    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433377    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433380    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433382    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433385    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433387    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433390    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433392    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433394    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433397    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433401    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433404    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433407    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433409    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433412    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433432    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433435    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 23:39:07.444678 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433827    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433832    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433835    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433838    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433841    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433844    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433847    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433850    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433853    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433855    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433858    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433860    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433862    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433865    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433867    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433870    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433873    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433875    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433878    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 23:39:07.445138 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433880    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433883    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433885    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433888    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433890    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433892    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433895    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433897    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433900    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433902    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433905    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433907    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433910    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433912    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433915    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433917    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433919    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433922    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433925    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433928    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 23:39:07.445614 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433932    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433935    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433941    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433944    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433947    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433949    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433952    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433955    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433957    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433960    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433962    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433965    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433968    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433971    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433973    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433977    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433979    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433982    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433984    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433987    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 23:39:07.446123 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433989    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433991    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433994    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433996    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.433998    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434001    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434004    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434008    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434011    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434014    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434016    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434019    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434024    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434027    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434031    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434034    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434037    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434039    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434042    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 23:39:07.446616 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434044    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434047    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434049    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434052    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434054    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434056    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434059    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434061    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434064    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434066    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434068    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434071    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434074    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434077    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434079    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434082    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434084    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434086    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434089    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434091    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 23:39:07.447112 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434094    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434179    2550 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434201    2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434211    2550 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434217    2550 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434224    2550 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434227    2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434233    2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434237    2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434240    2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434243    2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434246    2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434250    2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434253    2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434255    2550 flags.go:64] FLAG: --cgroup-root=""
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434258    2550 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434261    2550 flags.go:64] FLAG: --client-ca-file=""
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434264    2550 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434267    2550 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434271    2550 flags.go:64] FLAG: --cluster-domain=""
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434274    2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434277    2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434280    2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434283    2550 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 23:39:07.447631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434287    2550 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434290    2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434293    2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434296    2550 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434299    2550 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434302    2550 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434305    2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434308    2550 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434311    2550 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434315    2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434318    2550 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434321    2550 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434323    2550 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434326    2550 flags.go:64] FLAG: --enable-server="true"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434336    2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434344    2550 flags.go:64] FLAG: --event-burst="100"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434347    2550 flags.go:64] FLAG: --event-qps="50"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434350    2550 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434353    2550 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434355    2550 flags.go:64] FLAG: --eviction-hard=""
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434359    2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434362    2550 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434365    2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434378    2550 flags.go:64] FLAG: --eviction-soft=""
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434381    2550 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 23:39:07.448202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434383    2550 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434387    2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434389    2550 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434392    2550 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434395    2550 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434398    2550 flags.go:64] FLAG: --feature-gates=""
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434401    2550 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434404    2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434407    2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434430    2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434433    2550 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434436    2550 flags.go:64] FLAG: --help="false"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434439    2550 flags.go:64] FLAG: --hostname-override="ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434442    2550 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434445    2550 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434448    2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434451    2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434454    2550 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434457    2550 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434459    2550 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434464    2550 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434467    2550 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434477    2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434480    2550 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 23:39:07.448932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434483    2550 flags.go:64] FLAG: --kube-reserved=""
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434486    2550 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434488    2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434491    2550 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434494    2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434497    2550 flags.go:64] FLAG: --lock-file=""
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434499    2550 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434502    2550 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434505    2550 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434510    2550 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434513    2550 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434515    2550 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434518    2550 flags.go:64] FLAG: --logging-format="text"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434521    2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434524    2550 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434527    2550 flags.go:64] FLAG: --manifest-url=""
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434530    2550 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434534    2550 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434536    2550 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434540    2550 flags.go:64] FLAG: --max-pods="110"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434543    2550 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434546    2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434549    2550 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434552    2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434555    2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 23:39:07.449539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434557    2550 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434560    2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434567    2550 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434571    2550 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434574    2550 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434579    2550 flags.go:64] FLAG: --pod-cidr=""
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434582    2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434587    2550 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434590    2550 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434593    2550 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434596    2550 flags.go:64] FLAG: --port="10250"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434599    2550 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434602    2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05e5d423973ff35a3"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434605    2550 flags.go:64] FLAG: --qos-reserved=""
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434608    2550 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434611    2550 flags.go:64] FLAG: --register-node="true"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434614    2550 flags.go:64] FLAG: --register-with-taints=""
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434617    2550 flags.go:64] FLAG: --registry-burst="10"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434620    2550 flags.go:64] FLAG: --registry-qps="5"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434623    2550 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434625    2550 flags.go:64] FLAG: --reserved-memory=""
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434629    2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434632    2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434635    2550 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434637    2550 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 23:39:07.450124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434640    2550 flags.go:64] FLAG: --runonce="false"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434643    2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434646    2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434649    2550 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434651    2550 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434654    2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434657    2550 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434660    2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434663    2550 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434666    2550 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434670    2550 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434673    2550 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434676    2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434680    2550 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434683    2550 flags.go:64] FLAG: --system-cgroups=""
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434686    2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434691    2550 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434694    2550 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434696    2550 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434703    2550 flags.go:64] FLAG: --tls-min-version=""
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434705    2550 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434708    2550 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434711    2550 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434714    2550 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434716    2550 flags.go:64] FLAG: --v="2"
Jul 13 23:39:07.450753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434720    2550 flags.go:64] FLAG: --version="false"
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434724    2550 flags.go:64] FLAG: --vmodule=""
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434727    2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.434730    2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434848    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434852    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434855    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434858    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434860    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434862    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434866    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434868    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434871    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434873    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434875    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434878    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434881    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434884    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434887    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434889    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 23:39:07.451390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434894    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434896    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434899    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434902    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434904    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434906    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434909    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434911    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434914    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434916    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434919    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434921    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434923    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434926    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434928    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434940    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434944    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434948    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434951    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 23:39:07.451911 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434953    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434956    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434958    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434961    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434963    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434966    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434968    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434970    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434973    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434975    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434979    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434981    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434983    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434987    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434990    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434992    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434996    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.434998    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435001    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435003    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 23:39:07.452354 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435006    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435008    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435011    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435013    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435017    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435020    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435023    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435025    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435029    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435031    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435034    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435037    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435039    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435042    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435045    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435047    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435050    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435052    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435055    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 23:39:07.452858 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435058    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435061    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435064    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435069    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435071    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435074    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435079    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435082    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435084    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435087    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435089    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435092    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435095    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435097    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435100    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435103    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435106    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435108    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435110    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435113    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 23:39:07.453305 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435115    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435117    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435120    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435122    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.435124    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.435883    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.444983    2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.444997    2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.445010    2550 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.445017    2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445070    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445075    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445078    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445081    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445084    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 23:39:07.453908 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445087    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445090    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445093    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445095    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445098    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445101    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445104    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445106    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445109    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445112    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445114    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445116    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445119    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445121    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445124    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445126    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445128    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445131    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445133    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 23:39:07.454285 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445137    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445141    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445144    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445147    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445150    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445154    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445157    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445159    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445162    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445165    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445167    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445170    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445172    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445174    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445177    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445179    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445182    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445185    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445187    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445189    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 23:39:07.454883 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445192    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445194    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445197    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445199    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445202    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445204    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445206    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445208    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445211    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445213    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445216    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445218    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445220    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445223    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445225    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445228    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445230    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445233    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445236    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445239    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 23:39:07.455390 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445242    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445246    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445250    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445253    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445256    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445258    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445261    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445264    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445266    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445269    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445271    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445273    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445276    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445278    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445281    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445283    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445286    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445288    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445291    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445293    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 23:39:07.456042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445295    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445297    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445300    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445302    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445305    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445307    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445310    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445312    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445315    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445318    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445322    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445329    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445332    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445335    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 23:39:07.456681 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445338    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.445343    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445469    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445474    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445478    2550 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445481    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445483    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445486    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445489    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445491    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445494    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445497    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445499    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445502    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445504    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 23:39:07.457018 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445525    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445529    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445532    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445535    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445538    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445541    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445543    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445546    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445549    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445552    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445555    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445558    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445560    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445563    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445565    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445568    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445571    2550 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445573    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445575    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445578    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 23:39:07.457454 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445580    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445582    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445585    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445587    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445589    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445592    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445594    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445596    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445599    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445601    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445603    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445606    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445608    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445611    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445613    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445615    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445618    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445620    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445623    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445626    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 23:39:07.458602 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445628    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445630    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445633    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445636    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445638    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445641    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445643    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445647    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445652    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445655    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445657    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445660    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445663    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445666    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445669    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445671    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445674    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445676    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445679    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 23:39:07.459113 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445682    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445685    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445687    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445690    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445692    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445695    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445697    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445700    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445702    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445705    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445707    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445709    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445712    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445714    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445717    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445719    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445721    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445724    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445726    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 23:39:07.459590 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445730    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445733    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445736    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445739    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445742    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445745    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445748    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:07.445752    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.445757    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.445848    2550 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.448111    2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.449713    2550 server.go:1019] "Starting client certificate rotation"
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.449807    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 23:39:07.460035 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.449841    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 23:39:07.460367 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.452569    2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 23:39:07.460367 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.457135    2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 23:39:07.468192 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.468172    2550 log.go:25] "Validated CRI v1 runtime API"
Jul 13 23:39:07.470382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.470361    2550 log.go:25] "Validated CRI v1 image API"
Jul 13 23:39:07.471135 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.471120    2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 23:39:07.473636 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.473611    2550 fs.go:135] Filesystem UUIDs: map[2e29e50d-d61c-479c-9ae8-9257233ff745:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f3d2b745-e547-42dc-be9b-45099b8ab513:/dev/nvme0n1p3]
Jul 13 23:39:07.473710 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.473635    2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 23:39:07.478694 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.478584    2550 manager.go:216] Machine: {Timestamp:2026-07-13 23:39:07.477176302 +0000 UTC m=+0.365556394 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098047 MemoryCapacity:32812183552 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cb4eb29772089fe61cc97b03b73db SystemUUID:ec2cb4eb-2977-2089-fe61-cc97b03b73db BootID:b0d3f215-868f-4ab3-8adb-4d5a71b89d59 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406093824 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c4:79:5d:63:85 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c4:79:5d:63:85 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:a5:dc:d3:aa:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812183552 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 23:39:07.478694 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.478691    2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 23:39:07.478791 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.478762    2550 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 23:39:07.481205 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.481187    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 23:39:07.482226 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.482198    2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 23:39:07.482361 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.482230    2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-167.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 23:39:07.483132 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.483120    2550 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 23:39:07.483172 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.483136    2550 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 23:39:07.483172 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.483143    2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 23:39:07.483172 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.483158    2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 23:39:07.484950 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.484941    2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 23:39:07.488319 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488308    2550 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 23:39:07.488459 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488450    2550 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 23:39:07.488490 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488481    2550 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 23:39:07.488517 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488495    2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 23:39:07.488549 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488520    2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 23:39:07.488549 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488535    2550 kubelet.go:398] "Adding apiserver pod source"
Jul 13 23:39:07.488549 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488547    2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 23:39:07.488936 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488924    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 23:39:07.488988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.488948    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 23:39:07.493012 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.492995    2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 23:39:07.495554 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495541    2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 23:39:07.495612 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495595    2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 23:39:07.495760 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495749    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 23:39:07.495803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495764    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 23:39:07.495803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495770    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 23:39:07.495803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495775    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 23:39:07.495803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495783    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 23:39:07.495803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.495788    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 23:39:07.496468 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.496456    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 23:39:07.496508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.496470    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 23:39:07.496508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.496478    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 23:39:07.496508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.496483    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 23:39:07.496508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.496495    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 23:39:07.496631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.496513    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 23:39:07.497391 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.497379    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 23:39:07.497470 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.497393    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 23:39:07.499599 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.499549    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 23:39:07.499732 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.499705    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 23:39:07.501207 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.501193    2550 server.go:1268] "Started kubelet"
Jul 13 23:39:07.501359 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.501304    2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 23:39:07.501570 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.501348    2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 23:39:07.501570 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.501481    2550 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 23:39:07.502062 ip-10-0-137-167 systemd[1]: Started Kubernetes Kubelet.
Jul 13 23:39:07.503548 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.503531    2550 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 23:39:07.505272 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.505255    2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 23:39:07.507052 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.507035    2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 23:39:07.510086 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.510065    2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-167.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 23:39:07.511047 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.510027    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-167.ec2.internal.18c1fdc46d1873f7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-167.ec2.internal,UID:ip-10-0-137-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-167.ec2.internal,},FirstTimestamp:2026-07-13 23:39:07.501163511 +0000 UTC m=+0.389543584,LastTimestamp:2026-07-13 23:39:07.501163511 +0000 UTC m=+0.389543584,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-167.ec2.internal,}"
Jul 13 23:39:07.512233 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.512216    2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 23:39:07.512312 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.512242    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 23:39:07.512388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.512368    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:07.513019 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.512999    2550 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 23:39:07.513019 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513021    2550 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 23:39:07.513145 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513127    2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 23:39:07.513194 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513169    2550 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 23:39:07.513194 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513179    2550 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 23:39:07.513474 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513458    2550 factory.go:55] Registering systemd factory
Jul 13 23:39:07.513474 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513478    2550 factory.go:223] Registration of the systemd container factory successfully
Jul 13 23:39:07.513765 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513745    2550 factory.go:153] Registering CRI-O factory
Jul 13 23:39:07.513765 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513767    2550 factory.go:223] Registration of the crio container factory successfully
Jul 13 23:39:07.513922 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513819    2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 23:39:07.513922 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513844    2550 factory.go:103] Registering Raw factory
Jul 13 23:39:07.513922 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.513859    2550 manager.go:1196] Started watching for new ooms in manager
Jul 13 23:39:07.514481 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.514467    2550 manager.go:349] Starting recovery of all containers
Jul 13 23:39:07.515256 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.515228    2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-167.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 23:39:07.515367 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.515345    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 23:39:07.524737 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.524712    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zx7hq"
Jul 13 23:39:07.526596 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.526572    2550 manager.go:354] Recovery completed
Jul 13 23:39:07.531766 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.531750    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.533069 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.533053    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zx7hq"
Jul 13 23:39:07.533872 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.533850    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.533924 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.533879    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.533924 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.533889    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.534395 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.534373    2550 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 23:39:07.534395 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.534387    2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 23:39:07.534515 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.534403    2550 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 23:39:07.534515 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.534443    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.535097 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.535082    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.535170 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.535107    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.535170 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.535117    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.535321 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.535310    2550 policy_none.go:49] "None policy: Start"
Jul 13 23:39:07.535357 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.535326    2550 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 23:39:07.535357 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.535336    2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 23:39:07.535818 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.535734    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-167.ec2.internal.18c1fdc46f0b7028  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-167.ec2.internal,UID:ip-10-0-137-167.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-167.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-167.ec2.internal,},FirstTimestamp:2026-07-13 23:39:07.533865 +0000 UTC m=+0.422245073,LastTimestamp:2026-07-13 23:39:07.533865 +0000 UTC m=+0.422245073,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-167.ec2.internal,}"
Jul 13 23:39:07.536177 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.536162    2550 policy_none.go:47] "Start"
Jul 13 23:39:07.567967 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.567950    2550 manager.go:337] "Starting Device Plugin manager"
Jul 13 23:39:07.568054 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.567980    2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 23:39:07.568054 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.567989    2550 server.go:87] "Starting device plugin registration server"
Jul 13 23:39:07.568219 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.568207    2550 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 23:39:07.568252 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.568221    2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 23:39:07.568362 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.568340    2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 23:39:07.568528 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.568456    2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 23:39:07.568528 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.568467    2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 23:39:07.568969 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.568952    2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 23:39:07.569032 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.568997    2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:07.668752 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.668696    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.670843 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.669931    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.670843 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.669958    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.670843 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.669967    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.670843 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.669991    2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.678373 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.678326    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 23:39:07.679690 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.679676    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 23:39:07.679753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.679713    2550 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 23:39:07.679753 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.679733    2550 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 23:39:07.679837 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.679768    2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 23:39:07.681888 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.681871    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 23:39:07.682044 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.682032    2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.682084 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.682050    2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-167.ec2.internal\": node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:07.726548 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.726523    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:07.779982 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.779955    2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"]
Jul 13 23:39:07.780057 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.780033    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.780883 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.780869    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.780948 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.780904    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.780948 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.780918    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.783324 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.783311    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.783496 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.783482    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.783542 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.783509    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.784664 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.784648    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.784730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.784683    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.784730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.784688    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.784730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.784700    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.784730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.784705    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.784730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.784714    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.787061 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.787038    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.787061 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.787064    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 23:39:07.788817 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.788799    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 23:39:07.788895 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.788832    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 23:39:07.788895 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.788844    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeHasSufficientPID"
Jul 13 23:39:07.814541 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.814520    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15816fd0954517b7da05c28e845e91a4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-167.ec2.internal\" (UID: \"15816fd0954517b7da05c28e845e91a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.814627 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.814543    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e5885023e5edbdeb397822308bda472-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal\" (UID: \"1e5885023e5edbdeb397822308bda472\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.814627 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.814567    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e5885023e5edbdeb397822308bda472-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal\" (UID: \"1e5885023e5edbdeb397822308bda472\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.822985 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.822958    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-167.ec2.internal\" not found" node="ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.827174 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.827146    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:07.827258 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.827244    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-167.ec2.internal\" not found" node="ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.915255 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.915233    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e5885023e5edbdeb397822308bda472-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal\" (UID: \"1e5885023e5edbdeb397822308bda472\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.915332 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.915263    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e5885023e5edbdeb397822308bda472-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal\" (UID: \"1e5885023e5edbdeb397822308bda472\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.915332 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.915305    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15816fd0954517b7da05c28e845e91a4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-167.ec2.internal\" (UID: \"15816fd0954517b7da05c28e845e91a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.915438 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.915327    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1e5885023e5edbdeb397822308bda472-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal\" (UID: \"1e5885023e5edbdeb397822308bda472\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.915438 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.915329    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/15816fd0954517b7da05c28e845e91a4-config\") pod \"kube-apiserver-proxy-ip-10-0-137-167.ec2.internal\" (UID: \"15816fd0954517b7da05c28e845e91a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.915438 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:07.915349    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e5885023e5edbdeb397822308bda472-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal\" (UID: \"1e5885023e5edbdeb397822308bda472\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:07.927375 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:07.927335    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.028152 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.028121    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.125634 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.125609    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:08.128734 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.128716    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.130495 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.130318    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:08.229384 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.229311    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.329816 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.329794    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.394346 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.394326    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 23:39:08.430742 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.430706    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.448861 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.448829    2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 23:39:08.448984 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.448970    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 23:39:08.449049 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.449010    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 23:39:08.531479 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.531455    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.536607 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.536572    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 23:34:08 +0000 UTC" deadline="2027-12-21 03:20:14.587367664 +0000 UTC"
Jul 13 23:39:08.536607 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.536602    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12603h41m6.050768783s"
Jul 13 23:39:08.632243 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.632215    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.644529 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:08.644501    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15816fd0954517b7da05c28e845e91a4.slice/crio-a06b3e1a269a75defb90a579d33da22fddb867ffba0b5a35e3acdba76197a53d WatchSource:0}: Error finding container a06b3e1a269a75defb90a579d33da22fddb867ffba0b5a35e3acdba76197a53d: Status 404 returned error can't find the container with id a06b3e1a269a75defb90a579d33da22fddb867ffba0b5a35e3acdba76197a53d
Jul 13 23:39:08.644823 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:08.644800    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5885023e5edbdeb397822308bda472.slice/crio-5f8cec7fac0d4335d288bcbd90a50dbddbf96243586c1c1b67d8e106444082c1 WatchSource:0}: Error finding container 5f8cec7fac0d4335d288bcbd90a50dbddbf96243586c1c1b67d8e106444082c1: Status 404 returned error can't find the container with id 5f8cec7fac0d4335d288bcbd90a50dbddbf96243586c1c1b67d8e106444082c1
Jul 13 23:39:08.649891 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.649876    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 23:39:08.683389 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.683349    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal" event={"ID":"1e5885023e5edbdeb397822308bda472","Type":"ContainerStarted","Data":"5f8cec7fac0d4335d288bcbd90a50dbddbf96243586c1c1b67d8e106444082c1"}
Jul 13 23:39:08.684395 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.684379    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal" event={"ID":"15816fd0954517b7da05c28e845e91a4","Type":"ContainerStarted","Data":"a06b3e1a269a75defb90a579d33da22fddb867ffba0b5a35e3acdba76197a53d"}
Jul 13 23:39:08.732926 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.732905    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.833478 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:08.833430    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-167.ec2.internal\" not found"
Jul 13 23:39:08.871824 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.871806    2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 23:39:08.899517 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.899499    2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 23:39:08.913132 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.913114    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:08.927505 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.927486    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 23:39:08.928620 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.928608    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal"
Jul 13 23:39:08.936508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:08.936492    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 23:39:10.579770 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.579737    2550 apiserver.go:52] "Watching apiserver"
Jul 13 23:39:10.587623 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.587603    2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 23:39:10.588699 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.588660    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-4kw9l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal","openshift-multus/multus-kkdxd","openshift-multus/network-metrics-daemon-4mr6t","openshift-network-diagnostics/network-check-target-vp7qn","openshift-network-operator/iptables-alerter-86sj8","openshift-ovn-kubernetes/ovnkube-node-zrpr4","kube-system/global-pull-secret-syncer-4z7b5","kube-system/konnectivity-agent-m4zpw","kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd","openshift-cluster-node-tuning-operator/tuned-7t7mt","openshift-dns/node-resolver-6gp4k","openshift-image-registry/node-ca-rgzpl"]
Jul 13 23:39:10.591522 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.591502    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.593839    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594103    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594311    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594346    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wnhqt\""
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594354    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594535    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594597    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.594685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.594671    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 23:39:10.596427 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.596408    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r9wgm\""
Jul 13 23:39:10.596532 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.596412    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.596600 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.596529    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.597356 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.596888    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.598702 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.598684    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.599376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.599213    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:10.599376 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.599274    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:10.599676 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.599658    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 23:39:10.599787 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.599769    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xlffv\""
Jul 13 23:39:10.599846 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.599806    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.601149 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.601131    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:10.601237 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.601189    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:10.603912 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.603877    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 23:39:10.604512 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.604480    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.606468 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.606444    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.606653 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.606626    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 23:39:10.606740 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.606657    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fnkhm\""
Jul 13 23:39:10.606816 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.606801    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.607148 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.606938    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 23:39:10.607148 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.606972    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 23:39:10.607275 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.607197    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.608983 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.608962    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 23:39:10.609171 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.609152    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 23:39:10.609271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.609189    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.609578 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.609559    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vh992\""
Jul 13 23:39:10.611105 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.611089    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.611243 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.611220    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 23:39:10.611461 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.611447    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.611511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.611461    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vjqwz\""
Jul 13 23:39:10.612647 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.612623    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.613498 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.613476    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 23:39:10.614483 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614460    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-ovn\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614576 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614494    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-run-ovn-kubernetes\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614576 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614517    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovn-node-metrics-cert\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614691 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614593    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:10.614691 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614628    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 23:39:10.614691 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614635    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:10.614691 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614681    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614729    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614750    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gs2f2\""
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614760    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-node-log\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614782    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-cni-bin\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614818    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614842    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovnkube-config\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.614875 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614863    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83cb2404-9a86-4938-a281-f5fd33be5f71-hosts-file\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.615126 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614944    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-os-release\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.615126 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614976    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6xnv\" (UniqueName: \"kubernetes.io/projected/5deee0cd-e42a-4244-ae33-a0207f26dfb1-kube-api-access-g6xnv\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.615126 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.614999    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.615126 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615000    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.615126 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615000    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615126 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615090    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-log-socket\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615120    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drzpr\" (UniqueName: \"kubernetes.io/projected/d79c4dfe-ae6c-470f-b0a6-700348765f66-kube-api-access-drzpr\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615170    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-system-cni-dir\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615193    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615218    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615245    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-systemd-units\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615273    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-systemd\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615303    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-etc-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615331    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615332    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-env-overrides\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615409    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grmk9\" (UniqueName: \"kubernetes.io/projected/cf7a5c33-e26b-48a5-bb02-a50ec358a582-kube-api-access-grmk9\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615442    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cnibin\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615466    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-kubelet\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615506    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-slash\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615533    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-run-netns\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615567    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-var-lib-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615587    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83cb2404-9a86-4938-a281-f5fd33be5f71-tmp-dir\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.615998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615620    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjghg\" (UniqueName: \"kubernetes.io/projected/83cb2404-9a86-4938-a281-f5fd33be5f71-kube-api-access-zjghg\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.615998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615645    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d79c4dfe-ae6c-470f-b0a6-700348765f66-host\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.615998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615670    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d79c4dfe-ae6c-470f-b0a6-700348765f66-serviceca\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.615998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615696    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-cni-netd\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615717    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovnkube-script-lib\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.615998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.615744    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wtmc\" (UniqueName: \"kubernetes.io/projected/7426e531-b898-4126-ae24-cc3925a6c20a-kube-api-access-5wtmc\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:10.617237 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.617206    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kvcdm\""
Jul 13 23:39:10.617346 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.617330    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:10.617490 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.617475    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:10.617711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.617693    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.617788 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.617767    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:10.620193 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.620176    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.623365 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.623129    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 23:39:10.623365 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.623190    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pj4hj\""
Jul 13 23:39:10.635925 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.635887    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qns7g"
Jul 13 23:39:10.641885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.641866    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qns7g"
Jul 13 23:39:10.701627 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.701600    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 23:39:10.704769 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.704751    2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 23:39:10.716955 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.716931    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-system-cni-dir\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.717066 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.716970    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.717066 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.716997    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-systemd-units\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717066 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717021    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-systemd\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717066 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717043    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-env-overrides\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717066 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717048    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-system-cni-dir\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717061    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-systemd-units\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717066    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grmk9\" (UniqueName: \"kubernetes.io/projected/cf7a5c33-e26b-48a5-bb02-a50ec358a582-kube-api-access-grmk9\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717107    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-systemd\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717114    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-sys\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717143    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cnibin\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717168    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-kubelet\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717190    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-slash\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717207    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cnibin\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717213    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-run-netns\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717237    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d79c4dfe-ae6c-470f-b0a6-700348765f66-serviceca\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717243    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-kubelet\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717259    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-device-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717278    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-run-netns\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717285    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-kubelet\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717306    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-slash\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717418    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-daemon-config\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717439    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-tuned\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717454    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvrsh\" (UniqueName: \"kubernetes.io/projected/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-kube-api-access-gvrsh\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717472    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-multus-certs\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717491    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f86be0b-cc85-4452-b67d-a4d942ebfa31-kubelet-config\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717505    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717526    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-cni-binary-copy\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717541    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-cni-bin\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717543    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-env-overrides\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717581    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717584    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-cni-netd\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717615    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovnkube-script-lib\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717624    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-cni-netd\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717646    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wtmc\" (UniqueName: \"kubernetes.io/projected/7426e531-b898-4126-ae24-cc3925a6c20a-kube-api-access-5wtmc\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717731    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d79c4dfe-ae6c-470f-b0a6-700348765f66-serviceca\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.717988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717750    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysctl-d\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717768    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-socket-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717782    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-system-cni-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717802    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-netns\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717819    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-run-ovn-kubernetes\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717834    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717848    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrwdv\" (UniqueName: \"kubernetes.io/projected/e61d7bb8-a8a2-49bd-b912-830b92a26000-kube-api-access-vrwdv\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717875    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-k8s-cni-cncf-io\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717922    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-cni-multus\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717945    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-hostroot\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717971    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drzpr\" (UniqueName: \"kubernetes.io/projected/d79c4dfe-ae6c-470f-b0a6-700348765f66-kube-api-access-drzpr\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.717995    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-lib-modules\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.718032    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718036    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-run-ovn-kubernetes\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718049    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovnkube-script-lib\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718084    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d7bb8-a8a2-49bd-b912-830b92a26000-tmp\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.718701 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718156    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.718196    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:39:11.218176876 +0000 UTC m=+3.016077554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718227    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718253    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718342    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-cni-bin\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718374    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718403    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-cni-bin\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718429    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718451    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-os-release\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718479    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-log-socket\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718506    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-kubernetes\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718513    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718551    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83cb2404-9a86-4938-a281-f5fd33be5f71-hosts-file\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718555    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-log-socket\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718586    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718599    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5deee0cd-e42a-4244-ae33-a0207f26dfb1-os-release\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718613    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-registration-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718615    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83cb2404-9a86-4938-a281-f5fd33be5f71-hosts-file\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718641    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718660    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718687    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-etc-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718729    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83cb2404-9a86-4938-a281-f5fd33be5f71-tmp-dir\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718758    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-etc-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718787    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-host\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718806    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-cnibin\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718830    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-var-lib-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718857    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjghg\" (UniqueName: \"kubernetes.io/projected/83cb2404-9a86-4938-a281-f5fd33be5f71-kube-api-access-zjghg\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718881    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d79c4dfe-ae6c-470f-b0a6-700348765f66-host\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718890    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-var-lib-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718926    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-modprobe-d\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718965    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-run\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718963    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d79c4dfe-ae6c-470f-b0a6-700348765f66-host\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.718985    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-sys-fs\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.719848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719008    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f86be0b-cc85-4452-b67d-a4d942ebfa31-dbus\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719033    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-socket-dir-parent\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719043    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83cb2404-9a86-4938-a281-f5fd33be5f71-tmp-dir\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719055    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-var-lib-kubelet\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719087    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-etc-kubernetes\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719117    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5deee0cd-e42a-4244-ae33-a0207f26dfb1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719130    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysconfig\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719155    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b29359d-4586-477e-bc73-cb99ea0ad8e7-iptables-alerter-script\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719176    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-etc-selinux\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719190    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-conf-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719209    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbd8m\" (UniqueName: \"kubernetes.io/projected/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-kube-api-access-jbd8m\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719226    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-os-release\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719242    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-ovn\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovn-node-metrics-cert\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719281    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d603b112-cd8d-4348-849c-76050a6d3ee9-konnectivity-ca\") pod \"konnectivity-agent-m4zpw\" (UID: \"d603b112-cd8d-4348-849c-76050a6d3ee9\") " pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719299    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-ovn\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719385    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-systemd\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.720351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719412    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-cni-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719427    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysctl-conf\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719445    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-node-log\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719472    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovnkube-config\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719487    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b29359d-4586-477e-bc73-cb99ea0ad8e7-host-slash\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719503    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-node-log\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719507    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6xnv\" (UniqueName: \"kubernetes.io/projected/5deee0cd-e42a-4244-ae33-a0207f26dfb1-kube-api-access-g6xnv\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719592    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719609    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d603b112-cd8d-4348-849c-76050a6d3ee9-agent-certs\") pod \"konnectivity-agent-m4zpw\" (UID: \"d603b112-cd8d-4348-849c-76050a6d3ee9\") " pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719624    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dds7p\" (UniqueName: \"kubernetes.io/projected/2b29359d-4586-477e-bc73-cb99ea0ad8e7-kube-api-access-dds7p\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719703    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf7a5c33-e26b-48a5-bb02-a50ec358a582-run-openvswitch\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719798    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovnkube-config\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.720826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.719981    2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 23:39:10.723515 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.723497    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf7a5c33-e26b-48a5-bb02-a50ec358a582-ovn-node-metrics-cert\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.724302 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.724283    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:10.724399 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.724307    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:10.724399 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.724321    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:10.724513 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.724410    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:11.224394467 +0000 UTC m=+3.022295170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:10.726431 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.726404    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grmk9\" (UniqueName: \"kubernetes.io/projected/cf7a5c33-e26b-48a5-bb02-a50ec358a582-kube-api-access-grmk9\") pod \"ovnkube-node-zrpr4\" (UID: \"cf7a5c33-e26b-48a5-bb02-a50ec358a582\") " pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.727195 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.727157    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wtmc\" (UniqueName: \"kubernetes.io/projected/7426e531-b898-4126-ae24-cc3925a6c20a-kube-api-access-5wtmc\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:10.727275 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.727215    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drzpr\" (UniqueName: \"kubernetes.io/projected/d79c4dfe-ae6c-470f-b0a6-700348765f66-kube-api-access-drzpr\") pod \"node-ca-rgzpl\" (UID: \"d79c4dfe-ae6c-470f-b0a6-700348765f66\") " pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.727966 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.727941    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjghg\" (UniqueName: \"kubernetes.io/projected/83cb2404-9a86-4938-a281-f5fd33be5f71-kube-api-access-zjghg\") pod \"node-resolver-6gp4k\" (UID: \"83cb2404-9a86-4938-a281-f5fd33be5f71\") " pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.728806 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.728785    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6xnv\" (UniqueName: \"kubernetes.io/projected/5deee0cd-e42a-4244-ae33-a0207f26dfb1-kube-api-access-g6xnv\") pod \"multus-additional-cni-plugins-4kw9l\" (UID: \"5deee0cd-e42a-4244-ae33-a0207f26dfb1\") " pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.820167 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820140    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-modprobe-d\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820175    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-run\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820200    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-sys-fs\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820221    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f86be0b-cc85-4452-b67d-a4d942ebfa31-dbus\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820241    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-socket-dir-parent\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-var-lib-kubelet\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820276    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-run\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820283    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-sys-fs\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820282    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-etc-kubernetes\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820319    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-etc-kubernetes\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820318    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-modprobe-d\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820355    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-socket-dir-parent\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820379    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-var-lib-kubelet\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820399    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysconfig\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820421    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b29359d-4586-477e-bc73-cb99ea0ad8e7-iptables-alerter-script\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820437    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-etc-selinux\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820450    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-conf-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820464    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbd8m\" (UniqueName: \"kubernetes.io/projected/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-kube-api-access-jbd8m\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820482    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-os-release\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820509    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-conf-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820511    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7f86be0b-cc85-4452-b67d-a4d942ebfa31-dbus\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820536    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysconfig\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820545    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-etc-selinux\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820551    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-os-release\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820572    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d603b112-cd8d-4348-849c-76050a6d3ee9-konnectivity-ca\") pod \"konnectivity-agent-m4zpw\" (UID: \"d603b112-cd8d-4348-849c-76050a6d3ee9\") " pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820644    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-systemd\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820751    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-cni-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.820761 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820768    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysctl-conf\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820787    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b29359d-4586-477e-bc73-cb99ea0ad8e7-host-slash\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820796    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-systemd\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820812    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d603b112-cd8d-4348-849c-76050a6d3ee9-agent-certs\") pod \"konnectivity-agent-m4zpw\" (UID: \"d603b112-cd8d-4348-849c-76050a6d3ee9\") " pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820836    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dds7p\" (UniqueName: \"kubernetes.io/projected/2b29359d-4586-477e-bc73-cb99ea0ad8e7-kube-api-access-dds7p\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820848    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b29359d-4586-477e-bc73-cb99ea0ad8e7-host-slash\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820864    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-sys\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820866    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-cni-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820890    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-device-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820938    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-kubelet\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820961    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-daemon-config\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820960    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysctl-conf\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.820993    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b29359d-4586-477e-bc73-cb99ea0ad8e7-iptables-alerter-script\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821015    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-kubelet\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821002    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-device-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821056    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-tuned\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821070    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-sys\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.821604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821083    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvrsh\" (UniqueName: \"kubernetes.io/projected/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-kube-api-access-gvrsh\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821107    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-multus-certs\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821130    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f86be0b-cc85-4452-b67d-a4d942ebfa31-kubelet-config\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821152    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821177    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-cni-binary-copy\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821208    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d603b112-cd8d-4348-849c-76050a6d3ee9-konnectivity-ca\") pod \"konnectivity-agent-m4zpw\" (UID: \"d603b112-cd8d-4348-849c-76050a6d3ee9\") " pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821201    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-cni-bin\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821206    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-multus-certs\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysctl-d\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821273    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-cni-bin\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.821342    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821390    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-socket-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:10.821424    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:11.321392198 +0000 UTC m=+3.119292895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821462    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-system-cni-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821495    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-netns\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821501    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-sysctl-d\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821516    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-socket-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.822382 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821522    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-multus-daemon-config\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821557    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-netns\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821555    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7f86be0b-cc85-4452-b67d-a4d942ebfa31-kubelet-config\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821570    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-system-cni-dir\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821577    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrwdv\" (UniqueName: \"kubernetes.io/projected/e61d7bb8-a8a2-49bd-b912-830b92a26000-kube-api-access-vrwdv\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821607    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-k8s-cni-cncf-io\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821639    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-cni-multus\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821655    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-cni-binary-copy\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821683    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-var-lib-cni-multus\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821700    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-host-run-k8s-cni-cncf-io\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821735    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-hostroot\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821784    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-hostroot\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821763    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-lib-modules\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821831    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d7bb8-a8a2-49bd-b912-830b92a26000-tmp\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821866    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-kubernetes\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821885    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821920    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-lib-modules\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821953    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-registration-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.822930 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821923    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-registration-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821985    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-host\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821993    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.822001    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-cnibin\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.821982    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-kubernetes\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.822059    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-cnibin\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.822063    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e61d7bb8-a8a2-49bd-b912-830b92a26000-host\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.823563 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.823451    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e61d7bb8-a8a2-49bd-b912-830b92a26000-etc-tuned\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.824243 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.824214    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d7bb8-a8a2-49bd-b912-830b92a26000-tmp\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.826975 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.826958    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d603b112-cd8d-4348-849c-76050a6d3ee9-agent-certs\") pod \"konnectivity-agent-m4zpw\" (UID: \"d603b112-cd8d-4348-849c-76050a6d3ee9\") " pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.828160 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.828138    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbd8m\" (UniqueName: \"kubernetes.io/projected/02caab57-ed2b-44ed-b831-fe9aaaaeaba6-kube-api-access-jbd8m\") pod \"multus-kkdxd\" (UID: \"02caab57-ed2b-44ed-b831-fe9aaaaeaba6\") " pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:10.829345 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.829306    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dds7p\" (UniqueName: \"kubernetes.io/projected/2b29359d-4586-477e-bc73-cb99ea0ad8e7-kube-api-access-dds7p\") pod \"iptables-alerter-86sj8\" (UID: \"2b29359d-4586-477e-bc73-cb99ea0ad8e7\") " pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.829689 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.829653    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvrsh\" (UniqueName: \"kubernetes.io/projected/1bf08e4b-e4a2-4970-8a32-2bca8b845cad-kube-api-access-gvrsh\") pod \"aws-ebs-csi-driver-node-rdjcd\" (UID: \"1bf08e4b-e4a2-4970-8a32-2bca8b845cad\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.829845 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.829805    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrwdv\" (UniqueName: \"kubernetes.io/projected/e61d7bb8-a8a2-49bd-b912-830b92a26000-kube-api-access-vrwdv\") pod \"tuned-7t7mt\" (UID: \"e61d7bb8-a8a2-49bd-b912-830b92a26000\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.902726 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.902699    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:10.909351 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.909332    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6gp4k"
Jul 13 23:39:10.915839 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.915820    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rgzpl"
Jul 13 23:39:10.922486 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.922470    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4kw9l"
Jul 13 23:39:10.930154 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.930136    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:10.936779 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.936761    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-86sj8"
Jul 13 23:39:10.943815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.943799    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd"
Jul 13 23:39:10.950414 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.950396    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7t7mt"
Jul 13 23:39:10.955928 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:10.955912    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kkdxd"
Jul 13 23:39:11.224951 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.224879    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:11.224951 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.224931    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:11.225155 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.225026    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:11.225155 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.225062    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:11.225155 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.225078    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:11.225155 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.225088    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:11.225155 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.225093    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:39:12.225076688 +0000 UTC m=+4.022977392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:11.225155 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.225139    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:12.225115218 +0000 UTC m=+4.023015924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:11.326248 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.326218    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:11.326447 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.326396    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:11.326495 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.326466    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:12.326445008 +0000 UTC m=+4.124345688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:11.378890 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.378864    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 23:39:11.630637 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.630346    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7a5c33_e26b_48a5_bb02_a50ec358a582.slice/crio-f8d998a5c4076f95b21e77f2df18a11344d256ac5675dc1df2d9cebc6d4ed1df WatchSource:0}: Error finding container f8d998a5c4076f95b21e77f2df18a11344d256ac5675dc1df2d9cebc6d4ed1df: Status 404 returned error can't find the container with id f8d998a5c4076f95b21e77f2df18a11344d256ac5675dc1df2d9cebc6d4ed1df
Jul 13 23:39:11.632292 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.632267    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5deee0cd_e42a_4244_ae33_a0207f26dfb1.slice/crio-33376c1c7f319a8b941eefac4e292987a649edbef95518063d4fef35bff4c828 WatchSource:0}: Error finding container 33376c1c7f319a8b941eefac4e292987a649edbef95518063d4fef35bff4c828: Status 404 returned error can't find the container with id 33376c1c7f319a8b941eefac4e292987a649edbef95518063d4fef35bff4c828
Jul 13 23:39:11.633171 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.633150    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf08e4b_e4a2_4970_8a32_2bca8b845cad.slice/crio-4afa28abf51aaa60ce7a169d7207134d28873dfd5c045141cc8e558c979c6a4f WatchSource:0}: Error finding container 4afa28abf51aaa60ce7a169d7207134d28873dfd5c045141cc8e558c979c6a4f: Status 404 returned error can't find the container with id 4afa28abf51aaa60ce7a169d7207134d28873dfd5c045141cc8e558c979c6a4f
Jul 13 23:39:11.635075 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.634591    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd603b112_cd8d_4348_849c_76050a6d3ee9.slice/crio-5af614c28e1afa01517627d28bb604a9a78c0a3056b8896b710869ff3d4ba1a9 WatchSource:0}: Error finding container 5af614c28e1afa01517627d28bb604a9a78c0a3056b8896b710869ff3d4ba1a9: Status 404 returned error can't find the container with id 5af614c28e1afa01517627d28bb604a9a78c0a3056b8896b710869ff3d4ba1a9
Jul 13 23:39:11.637835 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.637818    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02caab57_ed2b_44ed_b831_fe9aaaaeaba6.slice/crio-ff9f1e665ef9924b9d00955b5552e5c437016e3011c869e6ba57ea0ced1bdc9f WatchSource:0}: Error finding container ff9f1e665ef9924b9d00955b5552e5c437016e3011c869e6ba57ea0ced1bdc9f: Status 404 returned error can't find the container with id ff9f1e665ef9924b9d00955b5552e5c437016e3011c869e6ba57ea0ced1bdc9f
Jul 13 23:39:11.638930 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.638459    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79c4dfe_ae6c_470f_b0a6_700348765f66.slice/crio-e47d50cd64ae3701419ce315beb01c57f96c0c417fba28b191c1f08f8f732b4b WatchSource:0}: Error finding container e47d50cd64ae3701419ce315beb01c57f96c0c417fba28b191c1f08f8f732b4b: Status 404 returned error can't find the container with id e47d50cd64ae3701419ce315beb01c57f96c0c417fba28b191c1f08f8f732b4b
Jul 13 23:39:11.639999 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.639976    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b29359d_4586_477e_bc73_cb99ea0ad8e7.slice/crio-175c8aa1e24b119cf2708896ed60e28648dcc31da85ee06e662a28df8b7d37e1 WatchSource:0}: Error finding container 175c8aa1e24b119cf2708896ed60e28648dcc31da85ee06e662a28df8b7d37e1: Status 404 returned error can't find the container with id 175c8aa1e24b119cf2708896ed60e28648dcc31da85ee06e662a28df8b7d37e1
Jul 13 23:39:11.640395 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:11.640348    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83cb2404_9a86_4938_a281_f5fd33be5f71.slice/crio-7b9c2b7e28f68a8663eb9c84273b73976f72533e028be077ba78026c051cc6af WatchSource:0}: Error finding container 7b9c2b7e28f68a8663eb9c84273b73976f72533e028be077ba78026c051cc6af: Status 404 returned error can't find the container with id 7b9c2b7e28f68a8663eb9c84273b73976f72533e028be077ba78026c051cc6af
Jul 13 23:39:11.644833 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.644809    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 23:34:10 +0000 UTC" deadline="2028-03-07 10:05:00.341613496 +0000 UTC"
Jul 13 23:39:11.644833 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.644831    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14458h25m48.696785234s"
Jul 13 23:39:11.771383 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.771248    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:11.771534 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:11.771513    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:11.780533 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.780504    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal" event={"ID":"15816fd0954517b7da05c28e845e91a4","Type":"ContainerStarted","Data":"ba4906817b04d2543418f1a53acf691a1ce16efa5f8ade69b01a2fe0fb4fbcd0"}
Jul 13 23:39:11.782007 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.781985    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7t7mt" event={"ID":"e61d7bb8-a8a2-49bd-b912-830b92a26000","Type":"ContainerStarted","Data":"a98da27c9b3d187cb3322280947e3ae7f761c7875e16222b2cffd7c0ecbb3705"}
Jul 13 23:39:11.782958 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.782934    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-86sj8" event={"ID":"2b29359d-4586-477e-bc73-cb99ea0ad8e7","Type":"ContainerStarted","Data":"175c8aa1e24b119cf2708896ed60e28648dcc31da85ee06e662a28df8b7d37e1"}
Jul 13 23:39:11.783962 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.783932    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6gp4k" event={"ID":"83cb2404-9a86-4938-a281-f5fd33be5f71","Type":"ContainerStarted","Data":"7b9c2b7e28f68a8663eb9c84273b73976f72533e028be077ba78026c051cc6af"}
Jul 13 23:39:11.785355 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.785334    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rgzpl" event={"ID":"d79c4dfe-ae6c-470f-b0a6-700348765f66","Type":"ContainerStarted","Data":"e47d50cd64ae3701419ce315beb01c57f96c0c417fba28b191c1f08f8f732b4b"}
Jul 13 23:39:11.786403 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.786383    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kkdxd" event={"ID":"02caab57-ed2b-44ed-b831-fe9aaaaeaba6","Type":"ContainerStarted","Data":"ff9f1e665ef9924b9d00955b5552e5c437016e3011c869e6ba57ea0ced1bdc9f"}
Jul 13 23:39:11.787374 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.787353    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4zpw" event={"ID":"d603b112-cd8d-4348-849c-76050a6d3ee9","Type":"ContainerStarted","Data":"5af614c28e1afa01517627d28bb604a9a78c0a3056b8896b710869ff3d4ba1a9"}
Jul 13 23:39:11.788536 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.788496    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerStarted","Data":"33376c1c7f319a8b941eefac4e292987a649edbef95518063d4fef35bff4c828"}
Jul 13 23:39:11.789568 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.789548    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd" event={"ID":"1bf08e4b-e4a2-4970-8a32-2bca8b845cad","Type":"ContainerStarted","Data":"4afa28abf51aaa60ce7a169d7207134d28873dfd5c045141cc8e558c979c6a4f"}
Jul 13 23:39:11.791947 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:11.791928    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"f8d998a5c4076f95b21e77f2df18a11344d256ac5675dc1df2d9cebc6d4ed1df"}
Jul 13 23:39:12.233013 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.232908    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:12.233013 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.232966    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:12.233247 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.233117    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:12.233247 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.233135    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:12.233247 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.233146    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:12.233247 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.233204    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:14.233186244 +0000 UTC m=+6.031086922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:12.233636 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.233617    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:12.233706 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.233675    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:39:14.233658602 +0000 UTC m=+6.031559284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:12.333889 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.333850    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:12.334054 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.334043    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:12.334128 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.334102    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:14.33408436 +0000 UTC m=+6.131985043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:12.645106 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.644971    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 23:34:10 +0000 UTC" deadline="2028-01-10 12:39:02.839525926 +0000 UTC"
Jul 13 23:39:12.645106 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.645031    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13092h59m50.194499329s"
Jul 13 23:39:12.773654 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.773620    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:12.773848 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.773741    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:12.773848 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.773822    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:12.774186 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:12.773893    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:12.810812 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.810774    2550 generic.go:356] "Generic (PLEG): container finished" podID="1e5885023e5edbdeb397822308bda472" containerID="fc600334e4653b0cebda199e33ab7c06d6ae478cafc252ee6269d3e771f1c271" exitCode=0
Jul 13 23:39:12.811992 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.811734    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal" event={"ID":"1e5885023e5edbdeb397822308bda472","Type":"ContainerDied","Data":"fc600334e4653b0cebda199e33ab7c06d6ae478cafc252ee6269d3e771f1c271"}
Jul 13 23:39:12.825658 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:12.825606    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-167.ec2.internal" podStartSLOduration=2.825587533 podStartE2EDuration="2.825587533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:11.79599595 +0000 UTC m=+3.593896655" watchObservedRunningTime="2026-07-13 23:39:12.825587533 +0000 UTC m=+4.623488232"
Jul 13 23:39:13.770727 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:13.770694    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:13.771181 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:13.770824    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:13.832915 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:13.832846    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal" event={"ID":"1e5885023e5edbdeb397822308bda472","Type":"ContainerStarted","Data":"260068ad7dea2927776ea2ed1c80a8c95353c172a1f956a23f0441398bf9d1f1"}
Jul 13 23:39:13.850951 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:13.850869    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-167.ec2.internal" podStartSLOduration=3.850846653 podStartE2EDuration="3.850846653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:13.850222243 +0000 UTC m=+5.648122966" watchObservedRunningTime="2026-07-13 23:39:13.850846653 +0000 UTC m=+5.648747354"
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:14.254506    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:14.254561    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.254678    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.254715    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.254736    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.254749    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.254752    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:39:18.254731473 +0000 UTC m=+10.052632155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:14.254874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.254797    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:18.254781694 +0000 UTC m=+10.052682385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:14.355804 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:14.355766    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:14.355977 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.355957    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:14.356066 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.356020    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:18.356001973 +0000 UTC m=+10.153902672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:14.774560 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:14.774522    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:14.775011 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.774622    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:14.775011 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:14.774945    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:14.775140 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:14.775037    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:15.771139 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:15.771103    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:15.771302 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:15.771215    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:16.771133 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:16.771098    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:16.771587 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:16.771232    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:16.771737 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:16.771717    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:16.771842 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:16.771819    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:17.770737 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:17.770706    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:17.770930 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:17.770818    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:18.287078 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:18.287047    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:18.287078 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:18.287086    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:18.287509 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.287205    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:18.287509 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.287213    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:18.287509 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.287290    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:39:26.287271397 +0000 UTC m=+18.085172088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:18.287638 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.287218    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:18.287638 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.287589    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:18.287711 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.287644    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:26.287628625 +0000 UTC m=+18.085529304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:18.387774 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:18.387640    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:18.387943 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.387792    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:18.387943 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.387874    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:26.387854196 +0000 UTC m=+18.185754884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:18.772860 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:18.772770    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:18.773030 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.772888    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:18.773246 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:18.773229    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:18.773337 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:18.773314    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:19.771355 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:19.770822    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:19.771355 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:19.770985    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:20.771048 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:20.771003    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:20.771212 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:20.771127    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:20.771639 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:20.771614    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:20.772019 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:20.771756    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:21.770828 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:21.770794    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:21.771006 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:21.770959    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:22.771182 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:22.771148    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:22.771680 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:22.771149    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:22.771680 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:22.771286    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:22.771680 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:22.771379    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:23.771295 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:23.771263    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:23.771735 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:23.771404    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:24.771925 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:24.770969    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:24.771925 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:24.771000    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:24.771925 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:24.771102    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:24.771925 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:24.771188    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:25.770684 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:25.770653    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:25.770823 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:25.770780    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:26.355866 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:26.355842    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:26.355866 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:26.355870    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:26.356388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.355995    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:26.356388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.356010    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:26.356388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.356018    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:26.356388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.356021    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:26.356388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.356064    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:42.356051992 +0000 UTC m=+34.153952681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:26.356388 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.356078    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:39:42.356071321 +0000 UTC m=+34.153971998 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:26.457086 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:26.457054    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:26.457211 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.457188    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:26.457248 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.457244    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:42.457229626 +0000 UTC m=+34.255130302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:26.771378 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:26.771344    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:26.771378 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:26.771344    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:26.771587 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.771553    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:26.772017 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:26.771985    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:27.771135 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:27.771098    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:27.771575 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:27.771235    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:28.771215 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:28.771181    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:28.771637 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:28.771295    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:28.771690 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:28.771672    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:28.771786 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:28.771766    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:29.770999 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:29.770965    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:29.771188 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:29.771112    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:30.771017 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.770512    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:30.771017 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:30.770802    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:30.771017 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.770877    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:30.771017 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:30.770986    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:30.866836 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.866807    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7t7mt" event={"ID":"e61d7bb8-a8a2-49bd-b912-830b92a26000","Type":"ContainerStarted","Data":"5b400e12b46bce7256081a83f8b7bc8c41aad54780f77ca6ef3235d8ba582c0f"}
Jul 13 23:39:30.868408 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.868378    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6gp4k" event={"ID":"83cb2404-9a86-4938-a281-f5fd33be5f71","Type":"ContainerStarted","Data":"760e7bc7f9d63218d0fc027d34e6157d9f7f11911e4906255f4f726f970cfbaa"}
Jul 13 23:39:30.869845 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.869819    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rgzpl" event={"ID":"d79c4dfe-ae6c-470f-b0a6-700348765f66","Type":"ContainerStarted","Data":"ab1fd70223d35a85fd617fe3410a23ad6d5e0ec44478812a1d3ebb6a56c6781a"}
Jul 13 23:39:30.871365 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.871343    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kkdxd" event={"ID":"02caab57-ed2b-44ed-b831-fe9aaaaeaba6","Type":"ContainerStarted","Data":"6f44bb033edabcb135a74fc214539e8355e4f1fbd8ffa761eae02496c8fb9dbc"}
Jul 13 23:39:30.873344 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.873316    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m4zpw" event={"ID":"d603b112-cd8d-4348-849c-76050a6d3ee9","Type":"ContainerStarted","Data":"bd6111457025239f80668d073f3e1b58ab888e6564142c7d38f441472ecf13e7"}
Jul 13 23:39:30.876173 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.876150    2550 generic.go:356] "Generic (PLEG): container finished" podID="5deee0cd-e42a-4244-ae33-a0207f26dfb1" containerID="c86038fecdd6bc29a2a50a3553e1308951aec757982ab2a491a4b1620d5050c5" exitCode=0
Jul 13 23:39:30.876267 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.876216    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerDied","Data":"c86038fecdd6bc29a2a50a3553e1308951aec757982ab2a491a4b1620d5050c5"}
Jul 13 23:39:30.877849 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.877830    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd" event={"ID":"1bf08e4b-e4a2-4970-8a32-2bca8b845cad","Type":"ContainerStarted","Data":"b92f85eaf929f4f7e07e6b7efadd31148ee4b3a8a11aacf7889980e269d808ab"}
Jul 13 23:39:30.879394 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.879369    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"473aeb5b9be7f06a9b72564d8adf37f3e8bb8b31f80d24374970b6620855daba"}
Jul 13 23:39:30.882319 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.882283    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7t7mt" podStartSLOduration=3.079400941 podStartE2EDuration="21.882270817s" totalImagesPullingTime="18.802869876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:30.88213798 +0000 UTC m=+22.680038698" watchObservedRunningTime="2026-07-13 23:39:30.882270817 +0000 UTC m=+22.680171520"
Jul 13 23:39:30.896596 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.896549    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rgzpl" podStartSLOduration=4.093384666 podStartE2EDuration="22.896536355s" totalImagesPullingTime="18.803151689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:30.896414648 +0000 UTC m=+22.694315346" watchObservedRunningTime="2026-07-13 23:39:30.896536355 +0000 UTC m=+22.694437056"
Jul 13 23:39:30.910972 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.910939    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6gp4k" podStartSLOduration=4.143005675 podStartE2EDuration="22.910925577s" totalImagesPullingTime="18.767919902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:30.910677654 +0000 UTC m=+22.708578352" watchObservedRunningTime="2026-07-13 23:39:30.910925577 +0000 UTC m=+22.708826275"
Jul 13 23:39:30.943300 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.943259    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m4zpw" podStartSLOduration=6.914333962 podStartE2EDuration="21.943245449s" totalImagesPullingTime="15.028911487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:30.94304643 +0000 UTC m=+22.740947130" watchObservedRunningTime="2026-07-13 23:39:30.943245449 +0000 UTC m=+22.741146360"
Jul 13 23:39:30.958410 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:30.958367    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kkdxd" podStartSLOduration=2.869053313 podStartE2EDuration="21.958354963s" totalImagesPullingTime="19.08930165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:30.957950098 +0000 UTC m=+22.755850896" watchObservedRunningTime="2026-07-13 23:39:30.958354963 +0000 UTC m=+22.756255660"
Jul 13 23:39:31.457882 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.457665    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:31.770947 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.770861    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:31.771105 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:31.771006    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:31.885409 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.885384    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:39:31.885858 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.885822    2550 generic.go:356] "Generic (PLEG): container finished" podID="cf7a5c33-e26b-48a5-bb02-a50ec358a582" containerID="ebb45c10983a8d51087d9ba062323a7022bc98fbb7eb0578fba6c81c3dfd02b9" exitCode=1
Jul 13 23:39:31.886011 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.885977    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"3932a10f23ac1f096279fec6d72506db62b2584138273a004c180f49d8bc5eff"}
Jul 13 23:39:31.886087 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.886073    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"ae4a90dbdcc4bb144111b45f82ac208c7403efebce35d3f971d645dda88405e2"}
Jul 13 23:39:31.886170 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.886126    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"d768453e174f80fe38ad422c84ac0585cd0bdf28dda2d1c8fbe0ff86e3a92230"}
Jul 13 23:39:31.886225 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.886169    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"d55982e538ce56f9278694bf2d99459d5545814a9b38676b464fce0329d82cbc"}
Jul 13 23:39:31.886225 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:31.886199    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerDied","Data":"ebb45c10983a8d51087d9ba062323a7022bc98fbb7eb0578fba6c81c3dfd02b9"}
Jul 13 23:39:32.111600 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.111579    2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 23:39:32.672426 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.672317    2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T23:39:32.111595645Z","UUID":"1456afde-51e5-4ae1-8da4-101c99bdc354","Handler":null,"Name":"","Endpoint":""}
Jul 13 23:39:32.674125 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.674095    2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 23:39:32.674243 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.674145    2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 23:39:32.770824 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.770790    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:32.771023 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:32.770943    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:32.771023 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.770983    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:32.771146 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:32.771095    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:32.890698 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.890660    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-86sj8" event={"ID":"2b29359d-4586-477e-bc73-cb99ea0ad8e7","Type":"ContainerStarted","Data":"50547dc08c45cae64e976a310d89364fac3e1f1a948d9b258fb0ec075e999615"}
Jul 13 23:39:32.893215 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.893181    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd" event={"ID":"1bf08e4b-e4a2-4970-8a32-2bca8b845cad","Type":"ContainerStarted","Data":"12362dd849b86c9dee268677e252ff9c4d034b8344be240fd8f7519d90368a19"}
Jul 13 23:39:32.904626 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:32.904577    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-86sj8" podStartSLOduration=5.103172451 podStartE2EDuration="23.904560672s" totalImagesPullingTime="18.801388221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:32.904272914 +0000 UTC m=+24.702173611" watchObservedRunningTime="2026-07-13 23:39:32.904560672 +0000 UTC m=+24.702461373"
Jul 13 23:39:33.771004 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:33.770926    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:33.771142 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:33.771042    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:33.897750 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:33.897714    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd" event={"ID":"1bf08e4b-e4a2-4970-8a32-2bca8b845cad","Type":"ContainerStarted","Data":"d3a4fd32357a4e8414ffe375fc4161ec5e6fcadfa735209de4eb5899c5d60a15"}
Jul 13 23:39:33.917850 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:33.917768    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rdjcd" podStartSLOduration=3.967860886 podStartE2EDuration="24.917747377s" totalImagesPullingTime="20.949886491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:33.915772992 +0000 UTC m=+25.713673672" watchObservedRunningTime="2026-07-13 23:39:33.917747377 +0000 UTC m=+25.715648076"
Jul 13 23:39:34.774599 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:34.774418    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:34.774833 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:34.774489    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:34.774833 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:34.774692    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:34.774833 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:34.774775    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:35.321629 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.321570    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:35.322115 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.322066    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:35.771344 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.771267    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:35.771490 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:35.771389    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:35.902541 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.902510    2550 generic.go:356] "Generic (PLEG): container finished" podID="5deee0cd-e42a-4244-ae33-a0207f26dfb1" containerID="e09e794995c74d5d2281171edd6d5088476b269624d94e9021fbae843309aabd" exitCode=0
Jul 13 23:39:35.902744 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.902598    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerDied","Data":"e09e794995c74d5d2281171edd6d5088476b269624d94e9021fbae843309aabd"}
Jul 13 23:39:35.905671 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.905652    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:39:35.906063 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.906020    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"52fba24d5ce99f9e88dc31be4d748d93a66338bae28003454cf6010d4a6dfb53"}
Jul 13 23:39:35.906715 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:35.906699    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m4zpw"
Jul 13 23:39:36.773796 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:36.773731    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:36.774234 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:36.773731    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:36.774234 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:36.773838    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:36.774234 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:36.773911    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:36.912485 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:36.912452    2550 generic.go:356] "Generic (PLEG): container finished" podID="5deee0cd-e42a-4244-ae33-a0207f26dfb1" containerID="386e31d628a15da60e028ea80a8886570782a653ec501bda855b9d1a27ff216d" exitCode=0
Jul 13 23:39:36.912631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:36.912521    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerDied","Data":"386e31d628a15da60e028ea80a8886570782a653ec501bda855b9d1a27ff216d"}
Jul 13 23:39:37.771063 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.770795    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:37.771207 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:37.771112    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:37.917070 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.917039    2550 generic.go:356] "Generic (PLEG): container finished" podID="5deee0cd-e42a-4244-ae33-a0207f26dfb1" containerID="733eb2ff5f7be1042ab08f877db764de5199bd2a04844ce5f4d3aee763c87d1a" exitCode=0
Jul 13 23:39:37.917568 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.917125    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerDied","Data":"733eb2ff5f7be1042ab08f877db764de5199bd2a04844ce5f4d3aee763c87d1a"}
Jul 13 23:39:37.920573 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.920555    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:39:37.920928 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.920893    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"df73486d2726e12d31cee7858c945e6b711311d76a60d87103cbfb47d1e0e9ad"}
Jul 13 23:39:37.921169 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.921146    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:37.921289 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.921214    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:37.921289 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.921259    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:37.921397 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.921366    2550 scope.go:117] "RemoveContainer" containerID="ebb45c10983a8d51087d9ba062323a7022bc98fbb7eb0578fba6c81c3dfd02b9"
Jul 13 23:39:37.937888 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.937868    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:37.938083 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:37.938068    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:39:38.778039 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:38.777418    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:38.778039 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:38.777461    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:38.778039 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:38.777544    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:38.778039 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:38.777981    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:38.927958 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:38.927936    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:39:38.928347 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:38.928320    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" event={"ID":"cf7a5c33-e26b-48a5-bb02-a50ec358a582","Type":"ContainerStarted","Data":"3a3237880de7e5247c4fdf3840959adcc0cecbf549cf74eef313a2d4fac44c13"}
Jul 13 23:39:38.960415 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:38.960359    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4" podStartSLOduration=12.086923897 podStartE2EDuration="30.960343233s" totalImagesPullingTime="18.873419336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:38.958650685 +0000 UTC m=+30.756551414" watchObservedRunningTime="2026-07-13 23:39:38.960343233 +0000 UTC m=+30.758243930"
Jul 13 23:39:39.036337 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:39.036252    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4z7b5"]
Jul 13 23:39:39.036477 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:39.036362    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:39.036477 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:39.036459    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:39.040211 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:39.040173    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vp7qn"]
Jul 13 23:39:39.040343 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:39.040225    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4mr6t"]
Jul 13 23:39:39.040343 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:39.040300    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:39.040448 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:39.040390    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:39.040448 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:39.040440    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:39.040539 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:39.040505    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:40.770799 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:40.770582    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:40.771253 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:40.770882    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:40.771253 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:40.770629    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:40.771253 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:40.770994    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:40.771253 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:40.770590    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:40.771253 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:40.771055    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:42.372300 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.372267    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:42.372300 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.372301    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:42.372888 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.372384    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:42.372888 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.372397    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 23:39:42.372888 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.372413    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 23:39:42.372888 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.372422    2550 projected.go:196] Error preparing data for projected volume kube-api-access-twp46 for pod openshift-network-diagnostics/network-check-target-vp7qn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:42.372888 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.372458    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46 podName:ae561399-54c8-4dfd-ac0e-9cca0d67a1b0 nodeName:}" failed. No retries permitted until 2026-07-13 23:40:14.372444685 +0000 UTC m=+66.170345374 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-twp46" (UniqueName: "kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46") pod "network-check-target-vp7qn" (UID: "ae561399-54c8-4dfd-ac0e-9cca0d67a1b0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 23:39:42.372888 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.372471    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs podName:7426e531-b898-4126-ae24-cc3925a6c20a nodeName:}" failed. No retries permitted until 2026-07-13 23:40:14.372465154 +0000 UTC m=+66.170365831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs") pod "network-metrics-daemon-4mr6t" (UID: "7426e531-b898-4126-ae24-cc3925a6c20a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 23:39:42.473100 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.473069    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:42.473238 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.473180    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:42.473238 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.473223    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret podName:7f86be0b-cc85-4452-b67d-a4d942ebfa31 nodeName:}" failed. No retries permitted until 2026-07-13 23:40:14.47321186 +0000 UTC m=+66.271112536 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret") pod "global-pull-secret-syncer-4z7b5" (UID: "7f86be0b-cc85-4452-b67d-a4d942ebfa31") : object "kube-system"/"original-pull-secret" not registered
Jul 13 23:39:42.773892 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.773869    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:42.774022 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.773876    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:42.774091 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.773972    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4z7b5" podUID="7f86be0b-cc85-4452-b67d-a4d942ebfa31"
Jul 13 23:39:42.774139 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.774069    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mr6t" podUID="7426e531-b898-4126-ae24-cc3925a6c20a"
Jul 13 23:39:42.774139 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.774127    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:42.774208 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:42.774194    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vp7qn" podUID="ae561399-54c8-4dfd-ac0e-9cca0d67a1b0"
Jul 13 23:39:42.940631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.940551    2550 generic.go:356] "Generic (PLEG): container finished" podID="5deee0cd-e42a-4244-ae33-a0207f26dfb1" containerID="6aa302f96d42299417a63a367061d711ea36d19556fb0fee9f45f6850626865e" exitCode=0
Jul 13 23:39:42.940631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:42.940588    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerDied","Data":"6aa302f96d42299417a63a367061d711ea36d19556fb0fee9f45f6850626865e"}
Jul 13 23:39:43.492124 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.492097    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-167.ec2.internal" event="NodeReady"
Jul 13 23:39:43.492638 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.492204    2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 23:39:43.525646 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.525620    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"]
Jul 13 23:39:43.543709 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.543683    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"]
Jul 13 23:39:43.543881 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.543828    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.546064 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.546037    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.546203 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.546094    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.546203 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.546105    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cr4xg\""
Jul 13 23:39:43.546203 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.546191    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 13 23:39:43.547179 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.547134    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 13 23:39:43.563681 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.563625    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-949bc8468-nx79s"]
Jul 13 23:39:43.564151 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.564113    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"
Jul 13 23:39:43.566325 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.566301    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.566435 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.566345    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w2dc8\""
Jul 13 23:39:43.567976 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.567955    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.584082 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.584061    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-676457fb64-74c8b"]
Jul 13 23:39:43.584181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.584168    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.586477 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.586457    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 23:39:43.586574 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.586561    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 23:39:43.586642 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.586628    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 23:39:43.586723 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.586708    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8vhjc\""
Jul 13 23:39:43.591643 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.591620    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 23:39:43.605453 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.605433    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"]
Jul 13 23:39:43.605565 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.605550    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.608260 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.608241    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 13 23:39:43.608414 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.608392    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 13 23:39:43.608513 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.608494    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.608751 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.608336    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pkrxj\""
Jul 13 23:39:43.609148 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.609129    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.609371 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.609354    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 13 23:39:43.609493 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.609472    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 13 23:39:43.625789 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.625767    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"]
Jul 13 23:39:43.625937 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.625922    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:43.627994 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.627972    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 13 23:39:43.627994 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.627980    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fb785\""
Jul 13 23:39:43.628131 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.628103    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.628209 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.628196    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.635746 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.635731    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-c8c6g"]
Jul 13 23:39:43.635850 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.635836    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.638059 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.638041    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 13 23:39:43.638163 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.638046    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.638163 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.638089    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 13 23:39:43.638163 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.638090    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6x4ph\""
Jul 13 23:39:43.638324 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.638171    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.651937 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.651919    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"]
Jul 13 23:39:43.652031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.651970    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.654218 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.654200    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 13 23:39:43.654461 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.654445    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 13 23:39:43.654548 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.654502    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-26cp6\""
Jul 13 23:39:43.654548 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.654521    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.654729 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.654706    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.659553 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.659538    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 13 23:39:43.670224 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.670202    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-8pn7m"]
Jul 13 23:39:43.670329 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.670314    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.672443 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.672426    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 13 23:39:43.672535 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.672475    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.672535 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.672498    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8dwp4\""
Jul 13 23:39:43.672912 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.672877    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.673033 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.673019    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 13 23:39:43.680966 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.680948    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lkzt\" (UniqueName: \"kubernetes.io/projected/cee06c03-b815-4b87-b0b1-dd2c86353e9b-kube-api-access-7lkzt\") pod \"volume-data-source-validator-78495597f6-6btql\" (UID: \"cee06c03-b815-4b87-b0b1-dd2c86353e9b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"
Jul 13 23:39:43.681055 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.680973    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681055 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.680997    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.681055 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681011    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-trusted-ca\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681055 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681026    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-image-registry-private-configuration\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681055 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681044    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-bound-sa-token\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681249 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681112    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w45p\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-kube-api-access-7w45p\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681249 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681146    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e361e2-266a-453a-932b-353d6377de9f-ca-trust-extracted\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681249 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681175    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.681249 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681194    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-installation-pull-secrets\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.681249 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8pdv\" (UniqueName: \"kubernetes.io/projected/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-kube-api-access-j8pdv\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.681428 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.681267    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-registry-certificates\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.682386 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.682373    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"]
Jul 13 23:39:43.682425 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.682398    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"]
Jul 13 23:39:43.682483 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.682471    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.684748 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.684730    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.684841 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.684795    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 13 23:39:43.684936 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.684917    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.685278 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.685258    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 13 23:39:43.685363 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.685310    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lzdv8\""
Jul 13 23:39:43.690092 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.690075    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 13 23:39:43.696784 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.696765    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5796w"]
Jul 13 23:39:43.696927 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.696894    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"
Jul 13 23:39:43.699873 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.699849    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.699986 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.699890    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fhp2n\""
Jul 13 23:39:43.700048 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.699894    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.703294 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.703250    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:43.717006 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.716979    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z8cxq"]
Jul 13 23:39:43.717114 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.717077    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.719216 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.719197    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 23:39:43.719314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.719229    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 23:39:43.719314 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.719244    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zw85q\""
Jul 13 23:39:43.736025 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.736007    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"]
Jul 13 23:39:43.736133 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.736037    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"]
Jul 13 23:39:43.736133 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.736117    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:43.738668 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.738651    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b7hbx\""
Jul 13 23:39:43.738752 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.738682    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 23:39:43.738752 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.738732    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 23:39:43.738752 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.738735    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 23:39:43.756327 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756272    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"]
Jul 13 23:39:43.756327 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756304    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"]
Jul 13 23:39:43.756327 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756324    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-676457fb64-74c8b"]
Jul 13 23:39:43.756511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756342    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-949bc8468-nx79s"]
Jul 13 23:39:43.756511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756358    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-8pn7m"]
Jul 13 23:39:43.756511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756375    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"]
Jul 13 23:39:43.756511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756387    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:43.756511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756392    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z8cxq"]
Jul 13 23:39:43.756730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756519    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5796w"]
Jul 13 23:39:43.756730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756570    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-c8c6g"]
Jul 13 23:39:43.756730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756602    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"]
Jul 13 23:39:43.756730 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.756621    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"]
Jul 13 23:39:43.758369 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.758354    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 13 23:39:43.758542 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.758526    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 13 23:39:43.758610 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.758588    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wq2sk\""
Jul 13 23:39:43.782564 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782544    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.782670 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782571    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.782670 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782603    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8pdv\" (UniqueName: \"kubernetes.io/projected/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-kube-api-access-j8pdv\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.782670 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782640    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.782812 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782670    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lkzt\" (UniqueName: \"kubernetes.io/projected/cee06c03-b815-4b87-b0b1-dd2c86353e9b-kube-api-access-7lkzt\") pod \"volume-data-source-validator-78495597f6-6btql\" (UID: \"cee06c03-b815-4b87-b0b1-dd2c86353e9b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"
Jul 13 23:39:43.782812 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782698    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.782812 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782715    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.782812 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782732    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-config\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782820    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-stats-auth\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782861    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-serving-cert\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782890    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv4m2\" (UniqueName: \"kubernetes.io/projected/9a3d6143-30ec-4d61-8402-ca0e2b300b24-kube-api-access-qv4m2\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782937    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-trusted-ca\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782953    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lrk7\" (UniqueName: \"kubernetes.io/projected/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-kube-api-access-4lrk7\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782972    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w45p\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-kube-api-access-7w45p\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.782993    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br9xv\" (UniqueName: \"kubernetes.io/projected/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-kube-api-access-br9xv\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783096    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e361e2-266a-453a-932b-353d6377de9f-ca-trust-extracted\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783115    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcdpv\" (UniqueName: \"kubernetes.io/projected/0bfd78e0-9f5d-40d2-ab9b-8d93973a9226-kube-api-access-jcdpv\") pod \"network-check-source-869dcfcdcd-68xxd\" (UID: \"0bfd78e0-9f5d-40d2-ab9b-8d93973a9226\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783145    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-config\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783170    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcx6s\" (UniqueName: \"kubernetes.io/projected/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-kube-api-access-hcx6s\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783198    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783221    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783268    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-registry-certificates\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783299    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8691f032-2217-4c94-b168-144e74a61f5b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjds5\" (UniqueName: \"kubernetes.io/projected/8691f032-2217-4c94-b168-144e74a61f5b-kube-api-access-pjds5\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.783302    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:43.783376 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783380    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-snapshots\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783397    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-serving-cert\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783420    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783427    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e361e2-266a-453a-932b-353d6377de9f-ca-trust-extracted\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783474    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.783479    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls podName:ae0a023b-56d5-4aa5-8b90-5acc5e7829b5 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.283458625 +0000 UTC m=+36.081359325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-slmmw" (UID: "ae0a023b-56d5-4aa5-8b90-5acc5e7829b5") : secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783553    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-trusted-ca\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.783570    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.783584    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-949bc8468-nx79s: secret "image-registry-tls" not found
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783597    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-bound-sa-token\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783625    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-image-registry-private-configuration\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.783639    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls podName:c1e361e2-266a-453a-932b-353d6377de9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.283623495 +0000 UTC m=+36.081524191 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls") pod "image-registry-949bc8468-nx79s" (UID: "c1e361e2-266a-453a-932b-353d6377de9f") : secret "image-registry-tls" not found
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783663    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783688    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-default-certificate\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783717    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g58pc\" (UniqueName: \"kubernetes.io/projected/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-kube-api-access-g58pc\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.783783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-tmp\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.784261 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783784    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-registry-certificates\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.784261 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783802    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-installation-pull-secrets\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.784261 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.783828    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8691f032-2217-4c94-b168-144e74a61f5b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.784356 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.784341    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-trusted-ca\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.786544 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.786512    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:43.787932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.787891    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-installation-pull-secrets\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.788009 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.787940    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-image-registry-private-configuration\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.790006 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.789987    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8pdv\" (UniqueName: \"kubernetes.io/projected/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-kube-api-access-j8pdv\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:43.793064 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.793041    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w45p\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-kube-api-access-7w45p\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.793402 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.793384    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lkzt\" (UniqueName: \"kubernetes.io/projected/cee06c03-b815-4b87-b0b1-dd2c86353e9b-kube-api-access-7lkzt\") pod \"volume-data-source-validator-78495597f6-6btql\" (UID: \"cee06c03-b815-4b87-b0b1-dd2c86353e9b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"
Jul 13 23:39:43.793472 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.793401    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-bound-sa-token\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:43.874503 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.874475    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"
Jul 13 23:39:43.884778 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.884756    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.884844 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.884784    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-config\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.884844 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.884802    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6136130f-72a5-4fa2-8624-a2013d9c2f9f-tmp-dir\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.884844 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.884820    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-stats-auth\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.885028 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885007    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-serving-cert\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.885072 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885054    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv4m2\" (UniqueName: \"kubernetes.io/projected/9a3d6143-30ec-4d61-8402-ca0e2b300b24-kube-api-access-qv4m2\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:43.885113 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885086    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-trusted-ca\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.885165 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885112    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lrk7\" (UniqueName: \"kubernetes.io/projected/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-kube-api-access-4lrk7\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.885165 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885140    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br9xv\" (UniqueName: \"kubernetes.io/projected/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-kube-api-access-br9xv\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.885269 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885172    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcdpv\" (UniqueName: \"kubernetes.io/projected/0bfd78e0-9f5d-40d2-ab9b-8d93973a9226-kube-api-access-jcdpv\") pod \"network-check-source-869dcfcdcd-68xxd\" (UID: \"0bfd78e0-9f5d-40d2-ab9b-8d93973a9226\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"
Jul 13 23:39:43.885269 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885202    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e8ab74-34a5-40ec-911e-e186ea2a05df-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:43.885269 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885236    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-config\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.885430 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885268    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcx6s\" (UniqueName: \"kubernetes.io/projected/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-kube-api-access-hcx6s\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.885430 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885306    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:43.885430 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885345    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8691f032-2217-4c94-b168-144e74a61f5b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.885430 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885382    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjds5\" (UniqueName: \"kubernetes.io/projected/8691f032-2217-4c94-b168-144e74a61f5b-kube-api-access-pjds5\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.885621 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885429    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:43.885621 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885457    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99cafe45-8468-4bf3-854b-476ff78603d4-cert\") pod \"ingress-canary-z8cxq\" (UID: \"99cafe45-8468-4bf3-854b-476ff78603d4\") " pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:43.885621 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885463    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-config\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.885621 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885513    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-snapshots\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.885621 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885552    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-serving-cert\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.885621 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885600    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6136130f-72a5-4fa2-8624-a2013d9c2f9f-config-volume\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.885888 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.885755    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.886156 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.886122    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-trusted-ca\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.886268 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.886234    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 23:39:43.886330 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.886282    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-snapshots\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.886330 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.886302    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.386283852 +0000 UTC m=+36.184184545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : secret "router-metrics-certs-default" not found
Jul 13 23:39:43.886610 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.886530    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.886610 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.886570    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-default-certificate\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.886711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.886676    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g58pc\" (UniqueName: \"kubernetes.io/projected/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-kube-api-access-g58pc\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.886768 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.886713    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddpwt\" (UniqueName: \"kubernetes.io/projected/6136130f-72a5-4fa2-8624-a2013d9c2f9f-kube-api-access-ddpwt\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.887024 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887000    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-tmp\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.887301 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887278    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-tmp\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.887375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887331    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8691f032-2217-4c94-b168-144e74a61f5b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.887375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887359    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.887474 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887385    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.887644 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887619    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-serving-cert\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887976    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.887983    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-serving-cert\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888022    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.888125    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.388108689 +0000 UTC m=+36.186009397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : configmap references non-existent config key: service-ca.crt
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888355    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8691f032-2217-4c94-b168-144e74a61f5b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888368    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-stats-auth\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.888444    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888476    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888495    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8691f032-2217-4c94-b168-144e74a61f5b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.888561    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls podName:9a3d6143-30ec-4d61-8402-ca0e2b300b24 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.388546396 +0000 UTC m=+36.186447086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls") pod "cluster-samples-operator-65db858894-6tlk4" (UID: "9a3d6143-30ec-4d61-8402-ca0e2b300b24") : secret "samples-operator-tls" not found
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888607    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8dtr\" (UniqueName: \"kubernetes.io/projected/99cafe45-8468-4bf3-854b-476ff78603d4-kube-api-access-s8dtr\") pod \"ingress-canary-z8cxq\" (UID: \"99cafe45-8468-4bf3-854b-476ff78603d4\") " pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888688    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-serving-cert\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.888815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.888782    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-config\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.889456 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.889438    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-default-certificate\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.895259 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.895232    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lrk7\" (UniqueName: \"kubernetes.io/projected/2f9d2f69-2b34-48e8-8a7d-b6ab20563f66-kube-api-access-4lrk7\") pod \"console-operator-7866497494-8pn7m\" (UID: \"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66\") " pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.895508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.895480    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv4m2\" (UniqueName: \"kubernetes.io/projected/9a3d6143-30ec-4d61-8402-ca0e2b300b24-kube-api-access-qv4m2\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:43.896625 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.896603    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g58pc\" (UniqueName: \"kubernetes.io/projected/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-kube-api-access-g58pc\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:43.897117 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.897094    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br9xv\" (UniqueName: \"kubernetes.io/projected/b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41-kube-api-access-br9xv\") pod \"service-ca-operator-7695fc6c8c-jz5n5\" (UID: \"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.897243 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.897131    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjds5\" (UniqueName: \"kubernetes.io/projected/8691f032-2217-4c94-b168-144e74a61f5b-kube-api-access-pjds5\") pod \"kube-storage-version-migrator-operator-799cb7f958-7xjhz\" (UID: \"8691f032-2217-4c94-b168-144e74a61f5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.897303 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.897274    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcx6s\" (UniqueName: \"kubernetes.io/projected/ed419e48-23c4-41c4-a1f1-37c3b07f7eba-kube-api-access-hcx6s\") pod \"insights-operator-68d9dd94cf-c8c6g\" (UID: \"ed419e48-23c4-41c4-a1f1-37c3b07f7eba\") " pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.898999 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.898978    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcdpv\" (UniqueName: \"kubernetes.io/projected/0bfd78e0-9f5d-40d2-ab9b-8d93973a9226-kube-api-access-jcdpv\") pod \"network-check-source-869dcfcdcd-68xxd\" (UID: \"0bfd78e0-9f5d-40d2-ab9b-8d93973a9226\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"
Jul 13 23:39:43.943333 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.943296    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"
Jul 13 23:39:43.945580 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.945548    2550 generic.go:356] "Generic (PLEG): container finished" podID="5deee0cd-e42a-4244-ae33-a0207f26dfb1" containerID="ecdfa0a24119f6e103a4ebe0fb6a1f90b496138a414c7df80ee588dd38353687" exitCode=0
Jul 13 23:39:43.945695 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.945611    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerDied","Data":"ecdfa0a24119f6e103a4ebe0fb6a1f90b496138a414c7df80ee588dd38353687"}
Jul 13 23:39:43.961247 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.961229    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g"
Jul 13 23:39:43.978655 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.978455    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"
Jul 13 23:39:43.990057 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.989958    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8dtr\" (UniqueName: \"kubernetes.io/projected/99cafe45-8468-4bf3-854b-476ff78603d4-kube-api-access-s8dtr\") pod \"ingress-canary-z8cxq\" (UID: \"99cafe45-8468-4bf3-854b-476ff78603d4\") " pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:43.990057 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990007    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6136130f-72a5-4fa2-8624-a2013d9c2f9f-tmp-dir\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.990057 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990054    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e8ab74-34a5-40ec-911e-e186ea2a05df-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990325    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990346    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6136130f-72a5-4fa2-8624-a2013d9c2f9f-tmp-dir\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990360    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99cafe45-8468-4bf3-854b-476ff78603d4-cert\") pod \"ingress-canary-z8cxq\" (UID: \"99cafe45-8468-4bf3-854b-476ff78603d4\") " pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990425    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6136130f-72a5-4fa2-8624-a2013d9c2f9f-config-volume\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990467    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.990497    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddpwt\" (UniqueName: \"kubernetes.io/projected/6136130f-72a5-4fa2-8624-a2013d9c2f9f-kube-api-access-ddpwt\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.990811    2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.990880    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert podName:c0e8ab74-34a5-40ec-911e-e186ea2a05df nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.490859567 +0000 UTC m=+36.288760260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wdhn7" (UID: "c0e8ab74-34a5-40ec-911e-e186ea2a05df") : secret "networking-console-plugin-cert" not found
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.991152    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:43.991206    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls podName:6136130f-72a5-4fa2-8624-a2013d9c2f9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:44.491189538 +0000 UTC m=+36.289090218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls") pod "dns-default-5796w" (UID: "6136130f-72a5-4fa2-8624-a2013d9c2f9f") : secret "dns-default-metrics-tls" not found
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.991241    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6136130f-72a5-4fa2-8624-a2013d9c2f9f-config-volume\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:43.991682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.991335    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:43.997373 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:43.996371    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99cafe45-8468-4bf3-854b-476ff78603d4-cert\") pod \"ingress-canary-z8cxq\" (UID: \"99cafe45-8468-4bf3-854b-476ff78603d4\") " pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:44.002719 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.002409    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8dtr\" (UniqueName: \"kubernetes.io/projected/99cafe45-8468-4bf3-854b-476ff78603d4-kube-api-access-s8dtr\") pod \"ingress-canary-z8cxq\" (UID: \"99cafe45-8468-4bf3-854b-476ff78603d4\") " pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:44.002719 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.002708    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e8ab74-34a5-40ec-911e-e186ea2a05df-nginx-conf\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:44.005193 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.005139    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddpwt\" (UniqueName: \"kubernetes.io/projected/6136130f-72a5-4fa2-8624-a2013d9c2f9f-kube-api-access-ddpwt\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:44.006594 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.006520    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"
Jul 13 23:39:44.045521 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.043866    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z8cxq"
Jul 13 23:39:44.077689 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.076576    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql"]
Jul 13 23:39:44.096936 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.096777    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee06c03_b815_4b87_b0b1_dd2c86353e9b.slice/crio-2e22aee33f40af409a10467f0549789d5958c2e1db0397b870ba7201d451575e WatchSource:0}: Error finding container 2e22aee33f40af409a10467f0549789d5958c2e1db0397b870ba7201d451575e: Status 404 returned error can't find the container with id 2e22aee33f40af409a10467f0549789d5958c2e1db0397b870ba7201d451575e
Jul 13 23:39:44.199109 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.198468    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz"]
Jul 13 23:39:44.202366 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.202335    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8691f032_2217_4c94_b168_144e74a61f5b.slice/crio-884a3cca9472bc5dc573d00eddd7f7c0420d1ccca6623267822f55e4b26ad8f2 WatchSource:0}: Error finding container 884a3cca9472bc5dc573d00eddd7f7c0420d1ccca6623267822f55e4b26ad8f2: Status 404 returned error can't find the container with id 884a3cca9472bc5dc573d00eddd7f7c0420d1ccca6623267822f55e4b26ad8f2
Jul 13 23:39:44.238854 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.238683    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-c8c6g"]
Jul 13 23:39:44.242439 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.241517    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded419e48_23c4_41c4_a1f1_37c3b07f7eba.slice/crio-e80f350e0464a5e8d69656f2be93933c575d74d6a0bf57839bfc41996ce3e8e6 WatchSource:0}: Error finding container e80f350e0464a5e8d69656f2be93933c575d74d6a0bf57839bfc41996ce3e8e6: Status 404 returned error can't find the container with id e80f350e0464a5e8d69656f2be93933c575d74d6a0bf57839bfc41996ce3e8e6
Jul 13 23:39:44.252371 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.252348    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-8pn7m"]
Jul 13 23:39:44.255707 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.255684    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f9d2f69_2b34_48e8_8a7d_b6ab20563f66.slice/crio-bbbbe201fc68f16d8216b1203e0d84efb040ec41096cd253cf0c532db7d62581 WatchSource:0}: Error finding container bbbbe201fc68f16d8216b1203e0d84efb040ec41096cd253cf0c532db7d62581: Status 404 returned error can't find the container with id bbbbe201fc68f16d8216b1203e0d84efb040ec41096cd253cf0c532db7d62581
Jul 13 23:39:44.261988 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.261966    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5"]
Jul 13 23:39:44.265391 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.265360    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00ca6dc_f835_4d36_8faa_a6b2e4e0ae41.slice/crio-bc90553a7f6a9cf3d43aaf599a6175fe9c0e08937d74bbcd2cfcd2e22634b6ab WatchSource:0}: Error finding container bc90553a7f6a9cf3d43aaf599a6175fe9c0e08937d74bbcd2cfcd2e22634b6ab: Status 404 returned error can't find the container with id bc90553a7f6a9cf3d43aaf599a6175fe9c0e08937d74bbcd2cfcd2e22634b6ab
Jul 13 23:39:44.273684 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.273661    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z8cxq"]
Jul 13 23:39:44.278750 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.278725    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd"]
Jul 13 23:39:44.281415 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.281392    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cafe45_8468_4bf3_854b_476ff78603d4.slice/crio-fc92f791f7380b7f94ec885ee00d23aafa06b540dba57dff6700f0ee8925f8d0 WatchSource:0}: Error finding container fc92f791f7380b7f94ec885ee00d23aafa06b540dba57dff6700f0ee8925f8d0: Status 404 returned error can't find the container with id fc92f791f7380b7f94ec885ee00d23aafa06b540dba57dff6700f0ee8925f8d0
Jul 13 23:39:44.282014 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:39:44.281993    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bfd78e0_9f5d_40d2_ab9b_8d93973a9226.slice/crio-3c2a92f85cd1ae7a0be46177215d18b4b45ee8dffebce511e5ac55853cfeca12 WatchSource:0}: Error finding container 3c2a92f85cd1ae7a0be46177215d18b4b45ee8dffebce511e5ac55853cfeca12: Status 404 returned error can't find the container with id 3c2a92f85cd1ae7a0be46177215d18b4b45ee8dffebce511e5ac55853cfeca12
Jul 13 23:39:44.293991 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.293970    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:44.294106 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.294034    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:44.294106 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.294084    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:44.294172 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.294124    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 23:39:44.294172 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.294133    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-949bc8468-nx79s: secret "image-registry-tls" not found
Jul 13 23:39:44.294172 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.294136    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls podName:ae0a023b-56d5-4aa5-8b90-5acc5e7829b5 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.294121252 +0000 UTC m=+37.092021928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-slmmw" (UID: "ae0a023b-56d5-4aa5-8b90-5acc5e7829b5") : secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:44.294172 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.294164    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls podName:c1e361e2-266a-453a-932b-353d6377de9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.294153747 +0000 UTC m=+37.092054468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls") pod "image-registry-949bc8468-nx79s" (UID: "c1e361e2-266a-453a-932b-353d6377de9f") : secret "image-registry-tls" not found
Jul 13 23:39:44.395202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.395176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:44.395279 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.395234    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:44.395325 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.395288    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:44.395358 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.395324    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.395303271 +0000 UTC m=+37.193203947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : configmap references non-existent config key: service-ca.crt
Jul 13 23:39:44.395401 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.395370    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 23:39:44.395433 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.395403    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.395392459 +0000 UTC m=+37.193293135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : secret "router-metrics-certs-default" not found
Jul 13 23:39:44.395433 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.395406    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 23:39:44.395495 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.395455    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls podName:9a3d6143-30ec-4d61-8402-ca0e2b300b24 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.395441284 +0000 UTC m=+37.193341986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls") pod "cluster-samples-operator-65db858894-6tlk4" (UID: "9a3d6143-30ec-4d61-8402-ca0e2b300b24") : secret "samples-operator-tls" not found
Jul 13 23:39:44.496056 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.496029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:44.496448 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.496082    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:44.496448 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.496160    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 23:39:44.496448 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.496191    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls podName:6136130f-72a5-4fa2-8624-a2013d9c2f9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.496181824 +0000 UTC m=+37.294082500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls") pod "dns-default-5796w" (UID: "6136130f-72a5-4fa2-8624-a2013d9c2f9f") : secret "dns-default-metrics-tls" not found
Jul 13 23:39:44.496448 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.496160    2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 23:39:44.496448 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:44.496237    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert podName:c0e8ab74-34a5-40ec-911e-e186ea2a05df nodeName:}" failed. No retries permitted until 2026-07-13 23:39:45.496222587 +0000 UTC m=+37.294123263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wdhn7" (UID: "c0e8ab74-34a5-40ec-911e-e186ea2a05df") : secret "networking-console-plugin-cert" not found
Jul 13 23:39:44.770687 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.770656    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:39:44.770882 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.770714    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:39:44.770882 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.770723    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:39:44.773278 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.773250    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 23:39:44.773413 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.773280    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5s29d\""
Jul 13 23:39:44.773413 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.773322    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 23:39:44.773413 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.773280    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qp66b\""
Jul 13 23:39:44.950545 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.950502    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pn7m" event={"ID":"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66","Type":"ContainerStarted","Data":"bbbbe201fc68f16d8216b1203e0d84efb040ec41096cd253cf0c532db7d62581"}
Jul 13 23:39:44.952702 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.952583    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd" event={"ID":"0bfd78e0-9f5d-40d2-ab9b-8d93973a9226","Type":"ContainerStarted","Data":"3c2a92f85cd1ae7a0be46177215d18b4b45ee8dffebce511e5ac55853cfeca12"}
Jul 13 23:39:44.954659 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.954595    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z8cxq" event={"ID":"99cafe45-8468-4bf3-854b-476ff78603d4","Type":"ContainerStarted","Data":"fc92f791f7380b7f94ec885ee00d23aafa06b540dba57dff6700f0ee8925f8d0"}
Jul 13 23:39:44.964040 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.963100    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" event={"ID":"5deee0cd-e42a-4244-ae33-a0207f26dfb1","Type":"ContainerStarted","Data":"5659b45e119f3e33bfe8e960e9ecaec65b089f8688fcbe53bf368c65641c3628"}
Jul 13 23:39:44.965693 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.965626    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5" event={"ID":"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41","Type":"ContainerStarted","Data":"bc90553a7f6a9cf3d43aaf599a6175fe9c0e08937d74bbcd2cfcd2e22634b6ab"}
Jul 13 23:39:44.968860 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.968813    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g" event={"ID":"ed419e48-23c4-41c4-a1f1-37c3b07f7eba","Type":"ContainerStarted","Data":"e80f350e0464a5e8d69656f2be93933c575d74d6a0bf57839bfc41996ce3e8e6"}
Jul 13 23:39:44.970703 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.970641    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz" event={"ID":"8691f032-2217-4c94-b168-144e74a61f5b","Type":"ContainerStarted","Data":"884a3cca9472bc5dc573d00eddd7f7c0420d1ccca6623267822f55e4b26ad8f2"}
Jul 13 23:39:44.973440 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.973416    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql" event={"ID":"cee06c03-b815-4b87-b0b1-dd2c86353e9b","Type":"ContainerStarted","Data":"2e22aee33f40af409a10467f0549789d5958c2e1db0397b870ba7201d451575e"}
Jul 13 23:39:44.985223 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:44.985132    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4kw9l" podStartSLOduration=8.341944854 podStartE2EDuration="36.98458916s" totalImagesPullingTime="28.642644306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:44.983660125 +0000 UTC m=+36.781560859" watchObservedRunningTime="2026-07-13 23:39:44.98458916 +0000 UTC m=+36.782489865"
Jul 13 23:39:45.106097 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.105993    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:45.206745 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.206633    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.303296    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.303411    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.303608    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.303624    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-949bc8468-nx79s: secret "image-registry-tls" not found
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.303684    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls podName:c1e361e2-266a-453a-932b-353d6377de9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.303665058 +0000 UTC m=+39.101565754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls") pod "image-registry-949bc8468-nx79s" (UID: "c1e361e2-266a-453a-932b-353d6377de9f") : secret "image-registry-tls" not found
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.304147    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:45.304872 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.304207    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls podName:ae0a023b-56d5-4aa5-8b90-5acc5e7829b5 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.304186757 +0000 UTC m=+39.102087455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-slmmw" (UID: "ae0a023b-56d5-4aa5-8b90-5acc5e7829b5") : secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:45.307465 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.307414    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.404771    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.404862    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.404997    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.405139    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.405202    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls podName:9a3d6143-30ec-4d61-8402-ca0e2b300b24 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.40518294 +0000 UTC m=+39.203083625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls") pod "cluster-samples-operator-65db858894-6tlk4" (UID: "9a3d6143-30ec-4d61-8402-ca0e2b300b24") : secret "samples-operator-tls" not found
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.405611    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.405662    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.405646073 +0000 UTC m=+39.203546769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : secret "router-metrics-certs-default" not found
Jul 13 23:39:45.405845 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.405732    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.405720696 +0000 UTC m=+39.203621379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : configmap references non-existent config key: service-ca.crt
Jul 13 23:39:45.505601 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.505468    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:45.505601 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:45.505567    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:45.506264 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.505788    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 23:39:45.506264 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.505857    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls podName:6136130f-72a5-4fa2-8624-a2013d9c2f9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.505838189 +0000 UTC m=+39.303738884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls") pod "dns-default-5796w" (UID: "6136130f-72a5-4fa2-8624-a2013d9c2f9f") : secret "dns-default-metrics-tls" not found
Jul 13 23:39:45.506396 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.506308    2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 23:39:45.506396 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:45.506371    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert podName:c0e8ab74-34a5-40ec-911e-e186ea2a05df nodeName:}" failed. No retries permitted until 2026-07-13 23:39:47.506350209 +0000 UTC m=+39.304250908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wdhn7" (UID: "c0e8ab74-34a5-40ec-911e-e186ea2a05df") : secret "networking-console-plugin-cert" not found
Jul 13 23:39:46.983004 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:46.982953    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql" event={"ID":"cee06c03-b815-4b87-b0b1-dd2c86353e9b","Type":"ContainerStarted","Data":"61f799358795fba3e08bcbd94bc30cfbdebfbd35dd0a8c84e7ba2d280a3a9312"}
Jul 13 23:39:46.998226 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:46.998167    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-6btql" podStartSLOduration=10.593991141 podStartE2EDuration="12.998145722s" totalImagesPullingTime="2.404154581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:46.996469113 +0000 UTC m=+38.794369812" watchObservedRunningTime="2026-07-13 23:39:46.998145722 +0000 UTC m=+38.796046421"
Jul 13 23:39:47.067404 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.067373    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:47.126046 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.125981    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:47.324333 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.324293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:47.324525 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.324396    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:47.324525 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.324466    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:47.324525 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.324505    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 23:39:47.324525 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.324520    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-949bc8468-nx79s: secret "image-registry-tls" not found
Jul 13 23:39:47.324683 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.324539    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls podName:ae0a023b-56d5-4aa5-8b90-5acc5e7829b5 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.324522097 +0000 UTC m=+43.122422773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-slmmw" (UID: "ae0a023b-56d5-4aa5-8b90-5acc5e7829b5") : secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:47.324683 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.324569    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls podName:c1e361e2-266a-453a-932b-353d6377de9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.324553392 +0000 UTC m=+43.122454094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls") pod "image-registry-949bc8468-nx79s" (UID: "c1e361e2-266a-453a-932b-353d6377de9f") : secret "image-registry-tls" not found
Jul 13 23:39:47.425173 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.425139    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:47.425347 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.425214    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:47.425347 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.425285    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:47.425347 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.425293    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 23:39:47.425511 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.425372    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.425353564 +0000 UTC m=+43.223254254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : configmap references non-existent config key: service-ca.crt
Jul 13 23:39:47.425511 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.425390    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.425382264 +0000 UTC m=+43.223282940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : secret "router-metrics-certs-default" not found
Jul 13 23:39:47.425511 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.425411    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 23:39:47.425511 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.425472    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls podName:9a3d6143-30ec-4d61-8402-ca0e2b300b24 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.425456361 +0000 UTC m=+43.223357055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls") pod "cluster-samples-operator-65db858894-6tlk4" (UID: "9a3d6143-30ec-4d61-8402-ca0e2b300b24") : secret "samples-operator-tls" not found
Jul 13 23:39:47.526292 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.526074    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:47.526477 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.526242    2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 23:39:47.526477 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:47.526364    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:47.526477 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.526418    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert podName:c0e8ab74-34a5-40ec-911e-e186ea2a05df nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.52639826 +0000 UTC m=+43.324298937 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wdhn7" (UID: "c0e8ab74-34a5-40ec-911e-e186ea2a05df") : secret "networking-console-plugin-cert" not found
Jul 13 23:39:47.526637 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.526485    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 23:39:47.526637 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:47.526528    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls podName:6136130f-72a5-4fa2-8624-a2013d9c2f9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:51.526516009 +0000 UTC m=+43.324416691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls") pod "dns-default-5796w" (UID: "6136130f-72a5-4fa2-8624-a2013d9c2f9f") : secret "dns-default-metrics-tls" not found
Jul 13 23:39:48.810435 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:48.810392    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:48.910353 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:48.910238    2550 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log" limit=1024
Jul 13 23:39:51.359388 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.359347    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:51.359843 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.359463    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:51.359843 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.359521    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:51.359843 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.359605    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls podName:ae0a023b-56d5-4aa5-8b90-5acc5e7829b5 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.359583727 +0000 UTC m=+51.157484416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-slmmw" (UID: "ae0a023b-56d5-4aa5-8b90-5acc5e7829b5") : secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:51.359843 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.359628    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 23:39:51.359843 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.359653    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-949bc8468-nx79s: secret "image-registry-tls" not found
Jul 13 23:39:51.359843 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.359712    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls podName:c1e361e2-266a-453a-932b-353d6377de9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.359696083 +0000 UTC m=+51.157596783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls") pod "image-registry-949bc8468-nx79s" (UID: "c1e361e2-266a-453a-932b-353d6377de9f") : secret "image-registry-tls" not found
Jul 13 23:39:51.460213 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.460165    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:51.460390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.460278    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:51.460390 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.460340    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 23:39:51.460505 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.460404    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls podName:9a3d6143-30ec-4d61-8402-ca0e2b300b24 nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.460388443 +0000 UTC m=+51.258289159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls") pod "cluster-samples-operator-65db858894-6tlk4" (UID: "9a3d6143-30ec-4d61-8402-ca0e2b300b24") : secret "samples-operator-tls" not found
Jul 13 23:39:51.460505 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.460443    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.460427833 +0000 UTC m=+51.258328524 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : configmap references non-existent config key: service-ca.crt
Jul 13 23:39:51.460626 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.460517    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 23:39:51.460626 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.460555    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.460542632 +0000 UTC m=+51.258443328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : secret "router-metrics-certs-default" not found
Jul 13 23:39:51.460626 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.460345    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:51.561442 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.561412    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:51.561587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:51.561470    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:51.561587 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.561549    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 23:39:51.561587 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.561557    2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 23:39:51.561704 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.561605    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert podName:c0e8ab74-34a5-40ec-911e-e186ea2a05df nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.561591666 +0000 UTC m=+51.359492344 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wdhn7" (UID: "c0e8ab74-34a5-40ec-911e-e186ea2a05df") : secret "networking-console-plugin-cert" not found
Jul 13 23:39:51.561704 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:51.561618    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls podName:6136130f-72a5-4fa2-8624-a2013d9c2f9f nodeName:}" failed. No retries permitted until 2026-07-13 23:39:59.561612396 +0000 UTC m=+51.359513072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls") pod "dns-default-5796w" (UID: "6136130f-72a5-4fa2-8624-a2013d9c2f9f") : secret "dns-default-metrics-tls" not found
Jul 13 23:39:53.004014 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.003974    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z8cxq" event={"ID":"99cafe45-8468-4bf3-854b-476ff78603d4","Type":"ContainerStarted","Data":"ff6434882d88b815aa3852fe2c0966047b09bd5cc764bced3eb19cc8a77a7b21"}
Jul 13 23:39:53.005970 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.005588    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5" event={"ID":"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41","Type":"ContainerStarted","Data":"6650a5f3569a11f9c84a2d9a03526c6b54edec1c5f51f75d42825462afd04aea"}
Jul 13 23:39:53.007181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.007146    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g" event={"ID":"ed419e48-23c4-41c4-a1f1-37c3b07f7eba","Type":"ContainerStarted","Data":"da234ea2a5311c5a957ebf85890d6e902b7777bfa2eda34bc94ac4ab6f2e4bb1"}
Jul 13 23:39:53.008583 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.008558    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz" event={"ID":"8691f032-2217-4c94-b168-144e74a61f5b","Type":"ContainerStarted","Data":"75d09b94fac2cfa69551794071d75ba2cd518012a7e20ccb223f806b9509d8f4"}
Jul 13 23:39:53.010654 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.010634    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/0.log"
Jul 13 23:39:53.010749 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.010668    2550 generic.go:356] "Generic (PLEG): container finished" podID="2f9d2f69-2b34-48e8-8a7d-b6ab20563f66" containerID="94e171b0579e14fafad2150e847385533d4c794b0ae9e89a3e1c658233c50b48" exitCode=255
Jul 13 23:39:53.010749 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.010734    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pn7m" event={"ID":"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66","Type":"ContainerDied","Data":"94e171b0579e14fafad2150e847385533d4c794b0ae9e89a3e1c658233c50b48"}
Jul 13 23:39:53.011008 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.010993    2550 scope.go:117] "RemoveContainer" containerID="94e171b0579e14fafad2150e847385533d4c794b0ae9e89a3e1c658233c50b48"
Jul 13 23:39:53.012585 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.012567    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd" event={"ID":"0bfd78e0-9f5d-40d2-ab9b-8d93973a9226","Type":"ContainerStarted","Data":"9e9e58259c410b1dc1c0f219e3332d0e10f1accbeac075b813ad7e73bd8211b3"}
Jul 13 23:39:53.020174 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.020124    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z8cxq" podStartSLOduration=2.223680704 podStartE2EDuration="10.020108481s" totalImagesPullingTime="7.796427777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:53.019490471 +0000 UTC m=+44.817391170" watchObservedRunningTime="2026-07-13 23:39:53.020108481 +0000 UTC m=+44.818009180"
Jul 13 23:39:53.035243 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.035197    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz" podStartSLOduration=11.16129391 podStartE2EDuration="19.035179148s" totalImagesPullingTime="7.873885238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:53.034542607 +0000 UTC m=+44.832443307" watchObservedRunningTime="2026-07-13 23:39:53.035179148 +0000 UTC m=+44.833079848"
Jul 13 23:39:53.051664 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.051227    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g" podStartSLOduration=11.215921654 podStartE2EDuration="19.051210031s" totalImagesPullingTime="7.835288377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:53.050592809 +0000 UTC m=+44.848493534" watchObservedRunningTime="2026-07-13 23:39:53.051210031 +0000 UTC m=+44.849110730"
Jul 13 23:39:53.067543 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.067259    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5" podStartSLOduration=11.254693706 podStartE2EDuration="19.067241198s" totalImagesPullingTime="7.812547492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:53.06604619 +0000 UTC m=+44.863946890" watchObservedRunningTime="2026-07-13 23:39:53.067241198 +0000 UTC m=+44.865141897"
Jul 13 23:39:53.099200 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.099147    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-68xxd" podStartSLOduration=11.303935993 podStartE2EDuration="19.09912598s" totalImagesPullingTime="7.795189987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:39:53.097889072 +0000 UTC m=+44.895789784" watchObservedRunningTime="2026-07-13 23:39:53.09912598 +0000 UTC m=+44.897026679"
Jul 13 23:39:53.734160 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.734128    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f7xwz"]
Jul 13 23:39:53.755495 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.755458    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f7xwz"]
Jul 13 23:39:53.755634 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.755572    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.757891 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.757866    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 23:39:53.758045 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.757974    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 23:39:53.758045 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.757989    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9c6sz\""
Jul 13 23:39:53.783279 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.783229    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9sdx\" (UniqueName: \"kubernetes.io/projected/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-api-access-j9sdx\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.783421 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.783309    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07a4af18-785a-4876-81d0-ef941ff8a4fe-crio-socket\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.783421 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.783348    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07a4af18-785a-4876-81d0-ef941ff8a4fe-data-volume\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.783421 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.783399    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.783581 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.783452    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.884249    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07a4af18-785a-4876-81d0-ef941ff8a4fe-crio-socket\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.884308    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07a4af18-785a-4876-81d0-ef941ff8a4fe-data-volume\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.884452    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.884544    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.884615    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07a4af18-785a-4876-81d0-ef941ff8a4fe-data-volume\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.884713    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9sdx\" (UniqueName: \"kubernetes.io/projected/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-api-access-j9sdx\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:53.884765    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:53.884835    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:39:54.38481557 +0000 UTC m=+46.182716250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.885122    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07a4af18-785a-4876-81d0-ef941ff8a4fe-crio-socket\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:53.885186    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 23:39:53.888429 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:53.885244    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:39:54.385221372 +0000 UTC m=+46.183122049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : configmap "kube-rbac-proxy" not found
Jul 13 23:39:53.896870 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.896846    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9sdx\" (UniqueName: \"kubernetes.io/projected/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-api-access-j9sdx\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:53.991668 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.991588    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:53.991808 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:53.991680    2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:39:54.016826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.016799    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:39:54.017273 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.017242    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/0.log"
Jul 13 23:39:54.017341 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.017275    2550 generic.go:356] "Generic (PLEG): container finished" podID="2f9d2f69-2b34-48e8-8a7d-b6ab20563f66" containerID="43381852f0e76e24b817e9f73be47d79284e3b76098606fca2277a62c7d17d15" exitCode=255
Jul 13 23:39:54.017391 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.017349    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pn7m" event={"ID":"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66","Type":"ContainerDied","Data":"43381852f0e76e24b817e9f73be47d79284e3b76098606fca2277a62c7d17d15"}
Jul 13 23:39:54.017457 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.017440    2550 scope.go:117] "RemoveContainer" containerID="94e171b0579e14fafad2150e847385533d4c794b0ae9e89a3e1c658233c50b48"
Jul 13 23:39:54.017649 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.017628    2550 scope.go:117] "RemoveContainer" containerID="43381852f0e76e24b817e9f73be47d79284e3b76098606fca2277a62c7d17d15"
Jul 13 23:39:54.017856 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:54.017828    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8pn7m_openshift-console-operator(2f9d2f69-2b34-48e8-8a7d-b6ab20563f66)\"" pod="openshift-console-operator/console-operator-7866497494-8pn7m" podUID="2f9d2f69-2b34-48e8-8a7d-b6ab20563f66"
Jul 13 23:39:54.391360 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.391326    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:54.391520 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:54.391376    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:54.391520 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:54.391481    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:54.391587 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:54.391550    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:39:55.391533106 +0000 UTC m=+47.189433785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:54.391587 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:54.391484    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 23:39:54.391669 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:54.391620    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:39:55.391607803 +0000 UTC m=+47.189508497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : configmap "kube-rbac-proxy" not found
Jul 13 23:39:55.021587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:55.021558    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:39:55.021947 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:55.021861    2550 scope.go:117] "RemoveContainer" containerID="43381852f0e76e24b817e9f73be47d79284e3b76098606fca2277a62c7d17d15"
Jul 13 23:39:55.022044 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:55.022025    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-8pn7m_openshift-console-operator(2f9d2f69-2b34-48e8-8a7d-b6ab20563f66)\"" pod="openshift-console-operator/console-operator-7866497494-8pn7m" podUID="2f9d2f69-2b34-48e8-8a7d-b6ab20563f66"
Jul 13 23:39:55.128017 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:55.127995    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6gp4k_83cb2404-9a86-4938-a281-f5fd33be5f71/dns-node-resolver/0.log"
Jul 13 23:39:55.399990 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:55.399956    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:55.400159 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:55.400092    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 23:39:55.400201 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:55.400096    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:55.400201 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:55.400168    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:55.400201 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:55.400184    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:39:57.400160735 +0000 UTC m=+49.198061433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : configmap "kube-rbac-proxy" not found
Jul 13 23:39:55.400311 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:55.400215    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:39:57.400203897 +0000 UTC m=+49.198104643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:56.528125 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:56.528098    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rgzpl_d79c4dfe-ae6c-470f-b0a6-700348765f66/node-ca/0.log"
Jul 13 23:39:57.131149 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:57.131113    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8cxq_99cafe45-8468-4bf3-854b-476ff78603d4/serve-healthcheck-canary/0.log"
Jul 13 23:39:57.418092 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:57.418014    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:57.418092 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:57.418062    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:57.418266 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:57.418176    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:57.418266 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:57.418249    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:40:01.418232937 +0000 UTC m=+53.216133616 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : secret "insights-runtime-extractor-tls" not found
Jul 13 23:39:57.418649 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:57.418631    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07a4af18-785a-4876-81d0-ef941ff8a4fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:39:59.435697 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.435665    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:39:59.436083 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.435725    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:39:59.436083 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.435804    2550 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:59.436083 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.435818    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 23:39:59.436083 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.435829    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-949bc8468-nx79s: secret "image-registry-tls" not found
Jul 13 23:39:59.436083 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.435874    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls podName:c1e361e2-266a-453a-932b-353d6377de9f nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.435857801 +0000 UTC m=+67.233758493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls") pod "image-registry-949bc8468-nx79s" (UID: "c1e361e2-266a-453a-932b-353d6377de9f") : secret "image-registry-tls" not found
Jul 13 23:39:59.436083 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.435886    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls podName:ae0a023b-56d5-4aa5-8b90-5acc5e7829b5 nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.435880218 +0000 UTC m=+67.233780894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-slmmw" (UID: "ae0a023b-56d5-4aa5-8b90-5acc5e7829b5") : secret "cluster-monitoring-operator-tls" not found
Jul 13 23:39:59.537029 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.537002    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:59.537147 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.537062    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:39:59.537196 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.537152    2550 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 13 23:39:59.537196 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.537157    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.537142201 +0000 UTC m=+67.335042890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : configmap references non-existent config key: service-ca.crt
Jul 13 23:39:59.537196 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.537187    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls podName:9a3d6143-30ec-4d61-8402-ca0e2b300b24 nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.537176093 +0000 UTC m=+67.335076770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls") pod "cluster-samples-operator-65db858894-6tlk4" (UID: "9a3d6143-30ec-4d61-8402-ca0e2b300b24") : secret "samples-operator-tls" not found
Jul 13 23:39:59.537318 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.537210    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:39:59.537318 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.537267    2550 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 13 23:39:59.537318 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.537287    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs podName:4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.537280689 +0000 UTC m=+67.335181365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs") pod "router-default-676457fb64-74c8b" (UID: "4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d") : secret "router-metrics-certs-default" not found
Jul 13 23:39:59.638206 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.638172    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:39:59.638355 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:39:59.638243    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:39:59.638355 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.638298    2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 13 23:39:59.638355 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.638337    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 23:39:59.638355 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.638345    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert podName:c0e8ab74-34a5-40ec-911e-e186ea2a05df nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.638332765 +0000 UTC m=+67.436233445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-wdhn7" (UID: "c0e8ab74-34a5-40ec-911e-e186ea2a05df") : secret "networking-console-plugin-cert" not found
Jul 13 23:39:59.638517 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:39:59.638380    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls podName:6136130f-72a5-4fa2-8624-a2013d9c2f9f nodeName:}" failed. No retries permitted until 2026-07-13 23:40:15.638366327 +0000 UTC m=+67.436267007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls") pod "dns-default-5796w" (UID: "6136130f-72a5-4fa2-8624-a2013d9c2f9f") : secret "dns-default-metrics-tls" not found
Jul 13 23:40:01.454442 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:01.454408    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:40:01.454874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:40:01.454558    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 23:40:01.454874 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:40:01.454624    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls podName:07a4af18-785a-4876-81d0-ef941ff8a4fe nodeName:}" failed. No retries permitted until 2026-07-13 23:40:09.454607828 +0000 UTC m=+61.252508533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7xwz" (UID: "07a4af18-785a-4876-81d0-ef941ff8a4fe") : secret "insights-runtime-extractor-tls" not found
Jul 13 23:40:03.991533 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:03.991495    2550 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:40:03.992032 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:03.991581    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:40:03.992032 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:03.991942    2550 scope.go:117] "RemoveContainer" containerID="43381852f0e76e24b817e9f73be47d79284e3b76098606fca2277a62c7d17d15"
Jul 13 23:40:04.051190 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:04.051173    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:40:05.055604 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:05.055578    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:40:05.055998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:05.055633    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-8pn7m" event={"ID":"2f9d2f69-2b34-48e8-8a7d-b6ab20563f66","Type":"ContainerStarted","Data":"860f642c7be47c0b338e4821055721a991984b76a8b1fc8a18a711628db4aa9f"}
Jul 13 23:40:05.055998 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:05.055924    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:40:05.060525 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:05.060506    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-8pn7m"
Jul 13 23:40:05.072754 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:05.072717    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-8pn7m" podStartSLOduration=23.246288998 podStartE2EDuration="31.072705356s" totalImagesPullingTime="7.826416358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:05.071775251 +0000 UTC m=+56.869675962" watchObservedRunningTime="2026-07-13 23:40:05.072705356 +0000 UTC m=+56.870606053"
Jul 13 23:40:09.529131 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:09.529085    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:40:09.531583 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:09.531559    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07a4af18-785a-4876-81d0-ef941ff8a4fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7xwz\" (UID: \"07a4af18-785a-4876-81d0-ef941ff8a4fe\") " pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:40:09.667994 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:09.667963    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9c6sz\""
Jul 13 23:40:09.676834 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:09.676813    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f7xwz"
Jul 13 23:40:09.804744 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:09.804672    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f7xwz"]
Jul 13 23:40:09.808034 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:09.808009    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a4af18_785a_4876_81d0_ef941ff8a4fe.slice/crio-03b8293e6ab1101cfc7345678e01d2ea50ccf992becb7024b5a33ed2d9cd41f4 WatchSource:0}: Error finding container 03b8293e6ab1101cfc7345678e01d2ea50ccf992becb7024b5a33ed2d9cd41f4: Status 404 returned error can't find the container with id 03b8293e6ab1101cfc7345678e01d2ea50ccf992becb7024b5a33ed2d9cd41f4
Jul 13 23:40:09.941881 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:09.941855    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zrpr4"
Jul 13 23:40:10.075243 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:10.075158    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7xwz" event={"ID":"07a4af18-785a-4876-81d0-ef941ff8a4fe","Type":"ContainerStarted","Data":"03b8293e6ab1101cfc7345678e01d2ea50ccf992becb7024b5a33ed2d9cd41f4"}
Jul 13 23:40:14.091276 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.091235    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7xwz" event={"ID":"07a4af18-785a-4876-81d0-ef941ff8a4fe","Type":"ContainerStarted","Data":"d7ca5864bdecf35a70732573db44e5f49bdfbd57c02375cdae592a062a7fa86e"}
Jul 13 23:40:14.375255 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.375210    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:40:14.375440 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.375277    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:40:14.377852 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.377657    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 23:40:14.378421 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.378394    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twp46\" (UniqueName: \"kubernetes.io/projected/ae561399-54c8-4dfd-ac0e-9cca0d67a1b0-kube-api-access-twp46\") pod \"network-check-target-vp7qn\" (UID: \"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0\") " pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:40:14.387658 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.387634    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7426e531-b898-4126-ae24-cc3925a6c20a-metrics-certs\") pod \"network-metrics-daemon-4mr6t\" (UID: \"7426e531-b898-4126-ae24-cc3925a6c20a\") " pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:40:14.476166 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.476133    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:40:14.478680 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.478656    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 23:40:14.488974 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.488953    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7f86be0b-cc85-4452-b67d-a4d942ebfa31-original-pull-secret\") pod \"global-pull-secret-syncer-4z7b5\" (UID: \"7f86be0b-cc85-4452-b67d-a4d942ebfa31\") " pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:40:14.529884 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.529863    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qp66b\""
Jul 13 23:40:14.538571 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.538554    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:40:14.539067 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.539041    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5s29d\""
Jul 13 23:40:14.546696 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.546670    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mr6t"
Jul 13 23:40:14.671860 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.671836    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vp7qn"]
Jul 13 23:40:14.674268 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:14.674241    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae561399_54c8_4dfd_ac0e_9cca0d67a1b0.slice/crio-b560a77f57f45d3870749dc6835400c9680699af595ef50164e125981b2a40f2 WatchSource:0}: Error finding container b560a77f57f45d3870749dc6835400c9680699af595ef50164e125981b2a40f2: Status 404 returned error can't find the container with id b560a77f57f45d3870749dc6835400c9680699af595ef50164e125981b2a40f2
Jul 13 23:40:14.690620 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.690399    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4mr6t"]
Jul 13 23:40:14.698838 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:14.698812    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7426e531_b898_4126_ae24_cc3925a6c20a.slice/crio-974e7b99c4a15cba786abfea9cfa10939aaba4168d213ab756af3b5af0f4ddb9 WatchSource:0}: Error finding container 974e7b99c4a15cba786abfea9cfa10939aaba4168d213ab756af3b5af0f4ddb9: Status 404 returned error can't find the container with id 974e7b99c4a15cba786abfea9cfa10939aaba4168d213ab756af3b5af0f4ddb9
Jul 13 23:40:14.783241 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.783217    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4z7b5"
Jul 13 23:40:14.903929 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:14.903836    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4z7b5"]
Jul 13 23:40:14.906482 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:14.906456    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f86be0b_cc85_4452_b67d_a4d942ebfa31.slice/crio-a6b0209bc5caac3e7cfa9d191d32a9d466221578e0fe1fb19c38384bf4cf8bbc WatchSource:0}: Error finding container a6b0209bc5caac3e7cfa9d191d32a9d466221578e0fe1fb19c38384bf4cf8bbc: Status 404 returned error can't find the container with id a6b0209bc5caac3e7cfa9d191d32a9d466221578e0fe1fb19c38384bf4cf8bbc
Jul 13 23:40:15.096829 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.096786    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4z7b5" event={"ID":"7f86be0b-cc85-4452-b67d-a4d942ebfa31","Type":"ContainerStarted","Data":"a6b0209bc5caac3e7cfa9d191d32a9d466221578e0fe1fb19c38384bf4cf8bbc"}
Jul 13 23:40:15.098390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.098318    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mr6t" event={"ID":"7426e531-b898-4126-ae24-cc3925a6c20a","Type":"ContainerStarted","Data":"974e7b99c4a15cba786abfea9cfa10939aaba4168d213ab756af3b5af0f4ddb9"}
Jul 13 23:40:15.100598 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.100566    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7xwz" event={"ID":"07a4af18-785a-4876-81d0-ef941ff8a4fe","Type":"ContainerStarted","Data":"11ca33d037070473bbc59c57145e84b8a2c534f3d0410920924c3059df993ac4"}
Jul 13 23:40:15.102307 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.102281    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vp7qn" event={"ID":"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0","Type":"ContainerStarted","Data":"95c846ee8209c74dac0d1575e6129078c74a0434c96e357fc76e1ca490ec4df6"}
Jul 13 23:40:15.102415 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.102327    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vp7qn" event={"ID":"ae561399-54c8-4dfd-ac0e-9cca0d67a1b0","Type":"ContainerStarted","Data":"b560a77f57f45d3870749dc6835400c9680699af595ef50164e125981b2a40f2"}
Jul 13 23:40:15.102473 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.102448    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:40:15.486168 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.485185    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:40:15.486168 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.485284    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:40:15.490511 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.490413    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"image-registry-949bc8468-nx79s\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") " pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:40:15.502602 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.502544    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae0a023b-56d5-4aa5-8b90-5acc5e7829b5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-slmmw\" (UID: \"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:40:15.587429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.586140    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:40:15.587429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.586226    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:15.587429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.586281    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:15.587429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.587027    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-service-ca-bundle\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:15.592307 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.592283    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d-metrics-certs\") pod \"router-default-676457fb64-74c8b\" (UID: \"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d\") " pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:15.592424 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.592379    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9a3d6143-30ec-4d61-8402-ca0e2b300b24-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-6tlk4\" (UID: \"9a3d6143-30ec-4d61-8402-ca0e2b300b24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:40:15.659727 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.659692    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cr4xg\""
Jul 13 23:40:15.667832 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.667806    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"
Jul 13 23:40:15.687284 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.687259    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:40:15.687398 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.687338    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:40:15.690062 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.690034    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e8ab74-34a5-40ec-911e-e186ea2a05df-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-wdhn7\" (UID: \"c0e8ab74-34a5-40ec-911e-e186ea2a05df\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:40:15.691834 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.691769    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6136130f-72a5-4fa2-8624-a2013d9c2f9f-metrics-tls\") pod \"dns-default-5796w\" (UID: \"6136130f-72a5-4fa2-8624-a2013d9c2f9f\") " pod="openshift-dns/dns-default-5796w"
Jul 13 23:40:15.695087 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.694868    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8vhjc\""
Jul 13 23:40:15.703985 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.703962    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:40:15.717491 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.717057    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-pkrxj\""
Jul 13 23:40:15.725685 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.725659    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:15.736135 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.736109    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fb785\""
Jul 13 23:40:15.744825 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.744595    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"
Jul 13 23:40:15.827770 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.827584    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zw85q\""
Jul 13 23:40:15.836801 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.836458    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5796w"
Jul 13 23:40:15.867148 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.866676    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wq2sk\""
Jul 13 23:40:15.875184 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.875031    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"
Jul 13 23:40:15.895459 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.894377    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vp7qn" podStartSLOduration=67.894246341 podStartE2EDuration="1m7.894246341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:15.118939377 +0000 UTC m=+66.916840094" watchObservedRunningTime="2026-07-13 23:40:15.894246341 +0000 UTC m=+67.692147042"
Jul 13 23:40:15.896635 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.896542    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw"]
Jul 13 23:40:15.903639 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:15.903456    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0a023b_56d5_4aa5_8b90_5acc5e7829b5.slice/crio-1d3ee653b53e22a2bd55181645fa9e60e7850f3565fd0a011cbfd3f40f84b9b6 WatchSource:0}: Error finding container 1d3ee653b53e22a2bd55181645fa9e60e7850f3565fd0a011cbfd3f40f84b9b6: Status 404 returned error can't find the container with id 1d3ee653b53e22a2bd55181645fa9e60e7850f3565fd0a011cbfd3f40f84b9b6
Jul 13 23:40:15.965941 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:15.961957    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-949bc8468-nx79s"]
Jul 13 23:40:15.996267 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:15.996227    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e361e2_266a_453a_932b_353d6377de9f.slice/crio-0106e529393bdd05f51af3266eed2c32fcddab422cf3ac9ac91589a8331c3438 WatchSource:0}: Error finding container 0106e529393bdd05f51af3266eed2c32fcddab422cf3ac9ac91589a8331c3438: Status 404 returned error can't find the container with id 0106e529393bdd05f51af3266eed2c32fcddab422cf3ac9ac91589a8331c3438
Jul 13 23:40:16.014088 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.014054    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-676457fb64-74c8b"]
Jul 13 23:40:16.019550 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.019454    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-mblkq"]
Jul 13 23:40:16.026627 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.025762    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:16.029649 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.029624    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 23:40:16.029867 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.029850    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 23:40:16.030258 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.030078    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rb9q7\""
Jul 13 23:40:16.031730 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:16.031675    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dbc64b3_d49c_4a9f_8242_eae3a0ac1e0d.slice/crio-9c8a31d5462c54e28b86c3b68f8b17da23ab368dc04e391aa5b4bdf0f7a59b07 WatchSource:0}: Error finding container 9c8a31d5462c54e28b86c3b68f8b17da23ab368dc04e391aa5b4bdf0f7a59b07: Status 404 returned error can't find the container with id 9c8a31d5462c54e28b86c3b68f8b17da23ab368dc04e391aa5b4bdf0f7a59b07
Jul 13 23:40:16.051307 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.051234    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-mblkq"]
Jul 13 23:40:16.096477 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.095704    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4w4d\" (UniqueName: \"kubernetes.io/projected/6b55064c-10d2-49eb-ae25-6838d1080089-kube-api-access-l4w4d\") pod \"downloads-68c9cbb497-mblkq\" (UID: \"6b55064c-10d2-49eb-ae25-6838d1080089\") " pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:16.113457 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.113433    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mr6t" event={"ID":"7426e531-b898-4126-ae24-cc3925a6c20a","Type":"ContainerStarted","Data":"f2e35747cefb662845365acc78849fbb752d70f92cf03820ed5573c2d7bd54c7"}
Jul 13 23:40:16.116396 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.116343    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw" event={"ID":"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5","Type":"ContainerStarted","Data":"1d3ee653b53e22a2bd55181645fa9e60e7850f3565fd0a011cbfd3f40f84b9b6"}
Jul 13 23:40:16.120763 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.120689    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5796w"]
Jul 13 23:40:16.125515 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.125490    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-676457fb64-74c8b" event={"ID":"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d","Type":"ContainerStarted","Data":"9c8a31d5462c54e28b86c3b68f8b17da23ab368dc04e391aa5b4bdf0f7a59b07"}
Jul 13 23:40:16.132046 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.132016    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-949bc8468-nx79s" event={"ID":"c1e361e2-266a-453a-932b-353d6377de9f","Type":"ContainerStarted","Data":"0106e529393bdd05f51af3266eed2c32fcddab422cf3ac9ac91589a8331c3438"}
Jul 13 23:40:16.142769 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.142702    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-wdhn7"]
Jul 13 23:40:16.149781 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:16.149750    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e8ab74_34a5_40ec_911e_e186ea2a05df.slice/crio-fb78d2e2fb18c7575266cd34d6c703bc18ec5fc4313ca6409642938e148ad050 WatchSource:0}: Error finding container fb78d2e2fb18c7575266cd34d6c703bc18ec5fc4313ca6409642938e148ad050: Status 404 returned error can't find the container with id fb78d2e2fb18c7575266cd34d6c703bc18ec5fc4313ca6409642938e148ad050
Jul 13 23:40:16.184740 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.184617    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4"]
Jul 13 23:40:16.196162 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.196099    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4w4d\" (UniqueName: \"kubernetes.io/projected/6b55064c-10d2-49eb-ae25-6838d1080089-kube-api-access-l4w4d\") pod \"downloads-68c9cbb497-mblkq\" (UID: \"6b55064c-10d2-49eb-ae25-6838d1080089\") " pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:16.206631 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.206603    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4w4d\" (UniqueName: \"kubernetes.io/projected/6b55064c-10d2-49eb-ae25-6838d1080089-kube-api-access-l4w4d\") pod \"downloads-68c9cbb497-mblkq\" (UID: \"6b55064c-10d2-49eb-ae25-6838d1080089\") " pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:16.374204 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.374159    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:16.548974 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:16.548937    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-mblkq"]
Jul 13 23:40:17.140500 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.139684    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mr6t" event={"ID":"7426e531-b898-4126-ae24-cc3925a6c20a","Type":"ContainerStarted","Data":"edc9db352ef42b24cab14bfe86c97e7f1de8d69bfb78113866d24b0dc4b37d07"}
Jul 13 23:40:17.142303 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.142228    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5796w" event={"ID":"6136130f-72a5-4fa2-8624-a2013d9c2f9f","Type":"ContainerStarted","Data":"9f16e856ad601931538111861c0bf00236b5984bc8eafcfe4693a9b399d7695d"}
Jul 13 23:40:17.144914 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.144830    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7" event={"ID":"c0e8ab74-34a5-40ec-911e-e186ea2a05df","Type":"ContainerStarted","Data":"fb78d2e2fb18c7575266cd34d6c703bc18ec5fc4313ca6409642938e148ad050"}
Jul 13 23:40:17.146852 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.146754    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-676457fb64-74c8b" event={"ID":"4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d","Type":"ContainerStarted","Data":"a491bb1703dab83ba50f9edab9df41cd5134b9be613d230c6a1c01ddf2b88819"}
Jul 13 23:40:17.148863 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.148838    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-949bc8468-nx79s" event={"ID":"c1e361e2-266a-453a-932b-353d6377de9f","Type":"ContainerStarted","Data":"2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7"}
Jul 13 23:40:17.149398 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.149377    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:40:17.151046 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.150972    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4" event={"ID":"9a3d6143-30ec-4d61-8402-ca0e2b300b24","Type":"ContainerStarted","Data":"76e724dd19672b49ccfdca688f64b5969c618843235abb39a40b62f1cf0cb23e"}
Jul 13 23:40:17.158256 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.157456    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4mr6t" podStartSLOduration=68.179623559 podStartE2EDuration="1m9.157437602s" totalImagesPullingTime="977.814043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:17.15640664 +0000 UTC m=+68.954307359" watchObservedRunningTime="2026-07-13 23:40:17.157437602 +0000 UTC m=+68.955338295"
Jul 13 23:40:17.177826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.177670    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-949bc8468-nx79s" podStartSLOduration=68.177651338 podStartE2EDuration="1m8.177651338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:17.176118951 +0000 UTC m=+68.974019675" watchObservedRunningTime="2026-07-13 23:40:17.177651338 +0000 UTC m=+68.975552037"
Jul 13 23:40:17.207963 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.207406    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-676457fb64-74c8b" podStartSLOduration=43.207384156 podStartE2EDuration="43.207384156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:17.19442324 +0000 UTC m=+68.992323920" watchObservedRunningTime="2026-07-13 23:40:17.207384156 +0000 UTC m=+69.005284857"
Jul 13 23:40:17.726827 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.726789    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:17.729442 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:17.729412    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:18.156186 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:18.156156    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:18.157560 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:18.157536    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-676457fb64-74c8b"
Jul 13 23:40:19.055386 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:19.055357    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b55064c_10d2_49eb_ae25_6838d1080089.slice/crio-8cc123486e339ae237ba57f1c139f0af241a9012a9da587c8678920218c1d423 WatchSource:0}: Error finding container 8cc123486e339ae237ba57f1c139f0af241a9012a9da587c8678920218c1d423: Status 404 returned error can't find the container with id 8cc123486e339ae237ba57f1c139f0af241a9012a9da587c8678920218c1d423
Jul 13 23:40:19.161104 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:19.161066    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-mblkq" event={"ID":"6b55064c-10d2-49eb-ae25-6838d1080089","Type":"ContainerStarted","Data":"8cc123486e339ae237ba57f1c139f0af241a9012a9da587c8678920218c1d423"}
Jul 13 23:40:23.178714 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.178672    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5796w" event={"ID":"6136130f-72a5-4fa2-8624-a2013d9c2f9f","Type":"ContainerStarted","Data":"da42f4296a9a0809eec064b4d5a6263d89850848d495c939287f789038c92ca1"}
Jul 13 23:40:23.179186 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.178730    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5796w" event={"ID":"6136130f-72a5-4fa2-8624-a2013d9c2f9f","Type":"ContainerStarted","Data":"8d6a38e07e02b67b477849962596595df80ab76e83a0583ffb0f870e7fa024df"}
Jul 13 23:40:23.179186 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.178969    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5796w"
Jul 13 23:40:23.181640 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.181615    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7xwz" event={"ID":"07a4af18-785a-4876-81d0-ef941ff8a4fe","Type":"ContainerStarted","Data":"43d52581f9ce41ac5961cf0f547a02881f692441cb41f138a2452b20d1922fb9"}
Jul 13 23:40:23.183387 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.183366    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw" event={"ID":"ae0a023b-56d5-4aa5-8b90-5acc5e7829b5","Type":"ContainerStarted","Data":"9218a562b7a461ac597fee73dbd33b6d753833f0d7d38b4b1e11fa776a1ff4c1"}
Jul 13 23:40:23.185041 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.185019    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7" event={"ID":"c0e8ab74-34a5-40ec-911e-e186ea2a05df","Type":"ContainerStarted","Data":"c496c8a3838725dfc7f7c6ed28d70beb519d76876d942c523ccd1f651af1c86e"}
Jul 13 23:40:23.186710 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.186665    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4z7b5" event={"ID":"7f86be0b-cc85-4452-b67d-a4d942ebfa31","Type":"ContainerStarted","Data":"5b1ccf88236bd287af9ae84381de2dccdc8302ea0b10627bacf68670737e56c4"}
Jul 13 23:40:23.188595 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.188572    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4" event={"ID":"9a3d6143-30ec-4d61-8402-ca0e2b300b24","Type":"ContainerStarted","Data":"d6b120ca9214ce5c247a72d1ec89aeb40eee498eecd32c57436584ccd06585e7"}
Jul 13 23:40:23.188708 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.188615    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4" event={"ID":"9a3d6143-30ec-4d61-8402-ca0e2b300b24","Type":"ContainerStarted","Data":"12f6ccd1893d750609a1d8701635f2f68e0a45f4b51317a08819f63829a7df72"}
Jul 13 23:40:23.199030 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.198957    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5796w" podStartSLOduration=33.92368921 podStartE2EDuration="40.198940779s" totalImagesPullingTime="6.275251569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:23.198581357 +0000 UTC m=+74.996482078" watchObservedRunningTime="2026-07-13 23:40:23.198940779 +0000 UTC m=+74.996841478"
Jul 13 23:40:23.213095 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.213049    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4z7b5" podStartSLOduration=66.716551858 podStartE2EDuration="1m14.213032367s" totalImagesPullingTime="7.496480509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:23.212041273 +0000 UTC m=+75.009941967" watchObservedRunningTime="2026-07-13 23:40:23.213032367 +0000 UTC m=+75.010933067"
Jul 13 23:40:23.230202 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.230158    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f7xwz" podStartSLOduration=17.744681866 podStartE2EDuration="30.23014009s" totalImagesPullingTime="12.485458224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 23:39:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:23.228752982 +0000 UTC m=+75.026653705" watchObservedRunningTime="2026-07-13 23:40:23.23014009 +0000 UTC m=+75.028040789"
Jul 13 23:40:23.243394 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.243358    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-slmmw" podStartSLOduration=42.755082549 podStartE2EDuration="49.24334573s" totalImagesPullingTime="6.488263181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:23.242551464 +0000 UTC m=+75.040452162" watchObservedRunningTime="2026-07-13 23:40:23.24334573 +0000 UTC m=+75.041246427"
Jul 13 23:40:23.260836 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.260790    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-wdhn7" podStartSLOduration=37.977491773 podStartE2EDuration="44.260775872s" totalImagesPullingTime="6.283284099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:23.259003989 +0000 UTC m=+75.056904713" watchObservedRunningTime="2026-07-13 23:40:23.260775872 +0000 UTC m=+75.058676570"
Jul 13 23:40:23.280564 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:23.280524    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-6tlk4" podStartSLOduration=43.086084319 podStartE2EDuration="49.280510664s" totalImagesPullingTime="6.194426345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:23.278865851 +0000 UTC m=+75.076766551" watchObservedRunningTime="2026-07-13 23:40:23.280510664 +0000 UTC m=+75.078411362"
Jul 13 23:40:31.575229 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.575094    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lkx8k"]
Jul 13 23:40:31.578241 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.578220    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.580865 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.580829    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 23:40:31.581284 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.581264    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 23:40:31.581501 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.581484    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 23:40:31.581770 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.581746    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 23:40:31.581966 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.581942    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7b4pc\""
Jul 13 23:40:31.631032 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.630996    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-tls\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631039    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-wtmp\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631062    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631117    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631152    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-metrics-client-ca\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631181 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631178    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-textfile\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631443 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631196    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdwkm\" (UniqueName: \"kubernetes.io/projected/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-kube-api-access-wdwkm\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631443 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631221    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-sys\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.631443 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.631243    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-root\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732164 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732131    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-tls\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732180    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-wtmp\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732206    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732238    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732281    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-metrics-client-ca\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732306    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-textfile\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732326 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:40:31.732312    2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732333    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdwkm\" (UniqueName: \"kubernetes.io/projected/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-kube-api-access-wdwkm\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732370    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-sys\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:40:31.732398    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-tls podName:a748fa36-27cd-4aa8-aeb6-b59617b0ea7f nodeName:}" failed. No retries permitted until 2026-07-13 23:40:32.232375659 +0000 UTC m=+84.030276358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-tls") pod "node-exporter-lkx8k" (UID: "a748fa36-27cd-4aa8-aeb6-b59617b0ea7f") : secret "node-exporter-tls" not found
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732427    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-sys\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732447    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-root\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732546    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-root\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.732605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.732562    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-wtmp\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.733227 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.733198    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.733347 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.733313    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-textfile\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.733404 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.733346    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-metrics-client-ca\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.735818 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.735788    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:31.746840 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:31.746812    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdwkm\" (UniqueName: \"kubernetes.io/projected/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-kube-api-access-wdwkm\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:32.237155 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:32.237120    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-tls\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:32.239971 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:32.239948    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a748fa36-27cd-4aa8-aeb6-b59617b0ea7f-node-exporter-tls\") pod \"node-exporter-lkx8k\" (UID: \"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f\") " pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:32.490442 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:32.490147    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lkx8k"
Jul 13 23:40:33.196471 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:33.196435    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5796w"
Jul 13 23:40:37.747655 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.747620    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 23:40:37.767178 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.767147    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.769403 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.769366    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 23:40:37.769921 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.769876    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 23:40:37.770368 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.770338    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 23:40:37.770558 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.770541    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 23:40:37.770717 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.770692    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 23:40:37.770790 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.770775    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 23:40:37.771375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771153    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 23:40:37.771375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771199    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 23:40:37.771375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771259    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 23:40:37.771547 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771381    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 23:40:37.771547 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771505    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 23:40:37.771653 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771566    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 23:40:37.771718 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771689    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-35qo0cg5uhp8p\""
Jul 13 23:40:37.771822 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771795    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 23:40:37.771822 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771795    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 23:40:37.771957 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771798    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p4zm8\""
Jul 13 23:40:37.771957 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.771798    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 23:40:37.775561 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.775542    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 23:40:37.784349 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784329    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-config\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784444 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784361    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784444 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784379    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e4f392f-07f6-4b16-acde-051c04c3de95-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784444 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784395    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784600 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784482    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784600 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784514    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784751 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784731    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.784931 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.784875    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785077 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785060    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e4f392f-07f6-4b16-acde-051c04c3de95-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785143 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785089    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785199 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785171    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785253 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785235    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785305 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785266    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ctv4\" (UniqueName: \"kubernetes.io/projected/9e4f392f-07f6-4b16-acde-051c04c3de95-kube-api-access-9ctv4\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785355 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785310    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785355 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785337    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785460 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785362    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785460 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785392    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785460 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785445    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785597 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785478    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.785597 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.785503    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.836947 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.836914    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-949bc8468-nx79s"]
Jul 13 23:40:37.842105 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.842077    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:40:37.886562 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886528    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e4f392f-07f6-4b16-acde-051c04c3de95-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886568    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886605    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886630    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886672    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886702    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886727    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e4f392f-07f6-4b16-acde-051c04c3de95-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886749    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886797    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886836    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.886885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886863    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ctv4\" (UniqueName: \"kubernetes.io/projected/9e4f392f-07f6-4b16-acde-051c04c3de95-kube-api-access-9ctv4\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886891    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886933    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886958    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.886990    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887048    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887096    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887121    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887432 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887160    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-config\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887432 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887193    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887432 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887329    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.887890 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.887710    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.889503 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.888344    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.889503 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.889197    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.889503 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.889386    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.889721 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.889580    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.892146 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.892124    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.892315 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.892289    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.893319 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.893106    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e4f392f-07f6-4b16-acde-051c04c3de95-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.894364 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.894335    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.894364 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.894350    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e4f392f-07f6-4b16-acde-051c04c3de95-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.894503 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.894386    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.894977 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.894888    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.895287 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.895263    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-config\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.895425 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.895346    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.895425 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.895388    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.895559 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.895483    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e4f392f-07f6-4b16-acde-051c04c3de95-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.896100 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.896072    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.896390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.896371    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e4f392f-07f6-4b16-acde-051c04c3de95-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:37.897650 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:37.897632    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ctv4\" (UniqueName: \"kubernetes.io/projected/9e4f392f-07f6-4b16-acde-051c04c3de95-kube-api-access-9ctv4\") pod \"prometheus-k8s-0\" (UID: \"9e4f392f-07f6-4b16-acde-051c04c3de95\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:38.080828 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:38.080790    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:40:39.276514 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:39.276378    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda748fa36_27cd_4aa8_aeb6_b59617b0ea7f.slice/crio-aae53cd765ca7d3663789506c4a1f8010337df60eab4f5ced60953968f611ccd WatchSource:0}: Error finding container aae53cd765ca7d3663789506c4a1f8010337df60eab4f5ced60953968f611ccd: Status 404 returned error can't find the container with id aae53cd765ca7d3663789506c4a1f8010337df60eab4f5ced60953968f611ccd
Jul 13 23:40:39.404345 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:39.404322    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 23:40:39.406789 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:40:39.406759    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4f392f_07f6_4b16_acde_051c04c3de95.slice/crio-9da01a7e9576bbdfc83df33dbfd8908c32fff282e20a73a1104c3469df03decb WatchSource:0}: Error finding container 9da01a7e9576bbdfc83df33dbfd8908c32fff282e20a73a1104c3469df03decb: Status 404 returned error can't find the container with id 9da01a7e9576bbdfc83df33dbfd8908c32fff282e20a73a1104c3469df03decb
Jul 13 23:40:40.267840 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:40.267789    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"9da01a7e9576bbdfc83df33dbfd8908c32fff282e20a73a1104c3469df03decb"}
Jul 13 23:40:40.269965 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:40.269934    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lkx8k" event={"ID":"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f","Type":"ContainerStarted","Data":"aae53cd765ca7d3663789506c4a1f8010337df60eab4f5ced60953968f611ccd"}
Jul 13 23:40:40.274530 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:40.274503    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-mblkq" event={"ID":"6b55064c-10d2-49eb-ae25-6838d1080089","Type":"ContainerStarted","Data":"5878551294ccf2031c53a59a4f2bf94cbd2a32a731ae23560b3ad299804c6c35"}
Jul 13 23:40:40.274633 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:40.274552    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:40.292603 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:40.292579    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-mblkq"
Jul 13 23:40:40.293580 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:40.293532    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-mblkq" podStartSLOduration=4.775381301 podStartE2EDuration="25.293514434s" totalImagesPullingTime="20.518133133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:40.292014715 +0000 UTC m=+92.089915434" watchObservedRunningTime="2026-07-13 23:40:40.293514434 +0000 UTC m=+92.091415133"
Jul 13 23:40:41.282863 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:41.281889    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"16123546ca56fbe42be14e02ad41caa65a289ecfea6525d8899f2ee48e024e27"}
Jul 13 23:40:41.284544 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:41.284516    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lkx8k" event={"ID":"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f","Type":"ContainerStarted","Data":"2f25d804d3310f9d60021aebdeb0e54eb5367fe136117b961f87f39c7940305c"}
Jul 13 23:40:42.290667 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:42.290636    2550 generic.go:356] "Generic (PLEG): container finished" podID="9e4f392f-07f6-4b16-acde-051c04c3de95" containerID="16123546ca56fbe42be14e02ad41caa65a289ecfea6525d8899f2ee48e024e27" exitCode=0
Jul 13 23:40:42.291231 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:42.290729    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerDied","Data":"16123546ca56fbe42be14e02ad41caa65a289ecfea6525d8899f2ee48e024e27"}
Jul 13 23:40:42.292508 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:42.292482    2550 generic.go:356] "Generic (PLEG): container finished" podID="a748fa36-27cd-4aa8-aeb6-b59617b0ea7f" containerID="2f25d804d3310f9d60021aebdeb0e54eb5367fe136117b961f87f39c7940305c" exitCode=0
Jul 13 23:40:42.292726 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:42.292589    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lkx8k" event={"ID":"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f","Type":"ContainerDied","Data":"2f25d804d3310f9d60021aebdeb0e54eb5367fe136117b961f87f39c7940305c"}
Jul 13 23:40:43.302130 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:43.302089    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lkx8k" event={"ID":"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f","Type":"ContainerStarted","Data":"2a002152582146fff1cba906e2a0860102ac273fe1a7d793f5649dfa86a98a90"}
Jul 13 23:40:43.302836 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:43.302157    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lkx8k" event={"ID":"a748fa36-27cd-4aa8-aeb6-b59617b0ea7f","Type":"ContainerStarted","Data":"605ba67c2b14f7b70abd39be9d85fee4026d6554d96edf215fd18659c2071d7b"}
Jul 13 23:40:43.321946 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:43.321879    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lkx8k" podStartSLOduration=10.43002821 podStartE2EDuration="12.321857444s" totalImagesPullingTime="1.795353822s" totalInitContainerRuntime="96.475412ms" isStatefulPod=true podCreationTimestamp="2026-07-13 23:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:43.319668578 +0000 UTC m=+95.117569303" watchObservedRunningTime="2026-07-13 23:40:43.321857444 +0000 UTC m=+95.119758154"
Jul 13 23:40:46.135523 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:46.135480    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vp7qn"
Jul 13 23:40:47.328075 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:47.327953    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"687cd587ae72f776fd6ef41b2da0ad8f6f71638802bf21f73a638b429d8493b4"}
Jul 13 23:40:47.328075 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:47.328042    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"171b51eb845c801c437c5f6ab24c0f572df74e7049ab64ba34ebcb125dd668eb"}
Jul 13 23:40:49.340658 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:49.340586    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"e03f81559ff44a5ca85227d5b97895b75a764d087563b06d3ecb562dfe85ea01"}
Jul 13 23:40:49.341014 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:49.340677    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"75fe2eec180532b54522b9f8467271b652e7051e3f6d923d8271bf3691cd2f5a"}
Jul 13 23:40:50.347692 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:50.347648    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"bd47f5dc85468f7e9a4cfd675590c22189b66736c24d6c225fc6e3583b7163da"}
Jul 13 23:40:50.348278 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:50.347718    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e4f392f-07f6-4b16-acde-051c04c3de95","Type":"ContainerStarted","Data":"c484976a174001f38dc86af1f7a84865ce2c626f22abc9d0297d6a4c33ec9833"}
Jul 13 23:40:50.374781 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:50.374727    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.165203715 podStartE2EDuration="13.374712196s" totalImagesPullingTime="8.18454173s" totalInitContainerRuntime="24.966751ms" isStatefulPod=false podCreationTimestamp="2026-07-13 23:40:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:40:50.374002187 +0000 UTC m=+102.171902905" watchObservedRunningTime="2026-07-13 23:40:50.374712196 +0000 UTC m=+102.172612894"
Jul 13 23:40:53.081436 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:40:53.081397    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:41:02.861012 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:02.860973    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-949bc8468-nx79s" podUID="c1e361e2-266a-453a-932b-353d6377de9f" containerName="registry" containerID="cri-o://2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7" gracePeriod=30
Jul 13 23:41:03.145346 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.145315    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:41:03.324001 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.323967    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-trusted-ca\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324208 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324009    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-installation-pull-secrets\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324208 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324027    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324208 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324070    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e361e2-266a-453a-932b-353d6377de9f-ca-trust-extracted\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324208 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324092    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-bound-sa-token\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324208 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324128    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-registry-certificates\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324208 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324198    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w45p\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-kube-api-access-7w45p\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324524 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324231    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-image-registry-private-configuration\") pod \"c1e361e2-266a-453a-932b-353d6377de9f\" (UID: \"c1e361e2-266a-453a-932b-353d6377de9f\") "
Jul 13 23:41:03.324597 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.324569    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 23:41:03.325405 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.325359    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 23:41:03.327105 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.327072    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:41:03.327200 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.327105    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 23:41:03.327200 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.327154    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 23:41:03.327200 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.327157    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-kube-api-access-7w45p" (OuterVolumeSpecName: "kube-api-access-7w45p") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "kube-api-access-7w45p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:41:03.327307 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.327195    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:41:03.332873 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.332851    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1e361e2-266a-453a-932b-353d6377de9f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c1e361e2-266a-453a-932b-353d6377de9f" (UID: "c1e361e2-266a-453a-932b-353d6377de9f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 23:41:03.397928 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.397874    2550 generic.go:356] "Generic (PLEG): container finished" podID="c1e361e2-266a-453a-932b-353d6377de9f" containerID="2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7" exitCode=0
Jul 13 23:41:03.398128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.397960    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-949bc8468-nx79s"
Jul 13 23:41:03.398128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.397973    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-949bc8468-nx79s" event={"ID":"c1e361e2-266a-453a-932b-353d6377de9f","Type":"ContainerDied","Data":"2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7"}
Jul 13 23:41:03.398128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.398036    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-949bc8468-nx79s" event={"ID":"c1e361e2-266a-453a-932b-353d6377de9f","Type":"ContainerDied","Data":"0106e529393bdd05f51af3266eed2c32fcddab422cf3ac9ac91589a8331c3438"}
Jul 13 23:41:03.398128 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.398065    2550 scope.go:117] "RemoveContainer" containerID="2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7"
Jul 13 23:41:03.399643 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.399622    2550 generic.go:356] "Generic (PLEG): container finished" podID="ed419e48-23c4-41c4-a1f1-37c3b07f7eba" containerID="da234ea2a5311c5a957ebf85890d6e902b7777bfa2eda34bc94ac4ab6f2e4bb1" exitCode=0
Jul 13 23:41:03.399732 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.399693    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g" event={"ID":"ed419e48-23c4-41c4-a1f1-37c3b07f7eba","Type":"ContainerDied","Data":"da234ea2a5311c5a957ebf85890d6e902b7777bfa2eda34bc94ac4ab6f2e4bb1"}
Jul 13 23:41:03.400067 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.400042    2550 scope.go:117] "RemoveContainer" containerID="da234ea2a5311c5a957ebf85890d6e902b7777bfa2eda34bc94ac4ab6f2e4bb1"
Jul 13 23:41:03.401326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.401287    2550 generic.go:356] "Generic (PLEG): container finished" podID="8691f032-2217-4c94-b168-144e74a61f5b" containerID="75d09b94fac2cfa69551794071d75ba2cd518012a7e20ccb223f806b9509d8f4" exitCode=0
Jul 13 23:41:03.401326 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.401323    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz" event={"ID":"8691f032-2217-4c94-b168-144e74a61f5b","Type":"ContainerDied","Data":"75d09b94fac2cfa69551794071d75ba2cd518012a7e20ccb223f806b9509d8f4"}
Jul 13 23:41:03.401579 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.401564    2550 scope.go:117] "RemoveContainer" containerID="75d09b94fac2cfa69551794071d75ba2cd518012a7e20ccb223f806b9509d8f4"
Jul 13 23:41:03.406668 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.406654    2550 scope.go:117] "RemoveContainer" containerID="2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7"
Jul 13 23:41:03.406968 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:41:03.406945    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7\": container with ID starting with 2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7 not found: ID does not exist" containerID="2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7"
Jul 13 23:41:03.407049 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.406974    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7"} err="failed to get container status \"2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7\": rpc error: code = NotFound desc = could not find container \"2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7\": container with ID starting with 2bbf73cbd44ba82ec20d4ce2a63174670dc5ad0773405e058535edcb666bbfb7 not found: ID does not exist"
Jul 13 23:41:03.425088 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425065    2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c1e361e2-266a-453a-932b-353d6377de9f-ca-trust-extracted\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425088 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425088    2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-bound-sa-token\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425268 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425099    2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-registry-certificates\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425268 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425107    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w45p\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-kube-api-access-7w45p\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425268 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425116    2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-image-registry-private-configuration\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425268 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425124    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c1e361e2-266a-453a-932b-353d6377de9f-trusted-ca\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425268 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425132    2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c1e361e2-266a-453a-932b-353d6377de9f-installation-pull-secrets\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.425268 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.425140    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c1e361e2-266a-453a-932b-353d6377de9f-registry-tls\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:41:03.450369 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.450037    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-949bc8468-nx79s"]
Jul 13 23:41:03.455854 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:03.455825    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-949bc8468-nx79s"]
Jul 13 23:41:04.061454 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:04.061426    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676457fb64-74c8b_4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d/router/0.log"
Jul 13 23:41:04.080735 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:04.080714    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8cxq_99cafe45-8468-4bf3-854b-476ff78603d4/serve-healthcheck-canary/0.log"
Jul 13 23:41:04.407585 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:04.407493    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-c8c6g" event={"ID":"ed419e48-23c4-41c4-a1f1-37c3b07f7eba","Type":"ContainerStarted","Data":"73b25b2c78d15a802decc9210867b47db0c9bc76a81507c1c2f530dba9e87fdb"}
Jul 13 23:41:04.409274 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:04.409251    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-7xjhz" event={"ID":"8691f032-2217-4c94-b168-144e74a61f5b","Type":"ContainerStarted","Data":"4823b98d43f8e2351fccc8011b7446855bd2c76e28344b5aa80e71c3439f6768"}
Jul 13 23:41:04.775738 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:04.775654    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e361e2-266a-453a-932b-353d6377de9f" path="/var/lib/kubelet/pods/c1e361e2-266a-453a-932b-353d6377de9f/volumes"
Jul 13 23:41:08.426324 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:08.426242    2550 generic.go:356] "Generic (PLEG): container finished" podID="b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41" containerID="6650a5f3569a11f9c84a2d9a03526c6b54edec1c5f51f75d42825462afd04aea" exitCode=0
Jul 13 23:41:08.426324 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:08.426313    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5" event={"ID":"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41","Type":"ContainerDied","Data":"6650a5f3569a11f9c84a2d9a03526c6b54edec1c5f51f75d42825462afd04aea"}
Jul 13 23:41:08.426706 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:08.426610    2550 scope.go:117] "RemoveContainer" containerID="6650a5f3569a11f9c84a2d9a03526c6b54edec1c5f51f75d42825462afd04aea"
Jul 13 23:41:09.431707 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:09.431666    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-jz5n5" event={"ID":"b00ca6dc-f835-4d36-8faa-a6b2e4e0ae41","Type":"ContainerStarted","Data":"3385d95cb44c9ee8d87f812a6695b490a2767357a5085bd3994e6910741e8d33"}
Jul 13 23:41:38.081390 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:38.081353    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:41:38.096806 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:38.096782    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:41:38.568516 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:41:38.568484    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 23:43:53.045475 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.045436    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"]
Jul 13 23:43:53.046471 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.046443    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e361e2-266a-453a-932b-353d6377de9f" containerName="registry"
Jul 13 23:43:53.046471 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.046473    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e361e2-266a-453a-932b-353d6377de9f" containerName="registry"
Jul 13 23:43:53.046612 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.046594    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e361e2-266a-453a-932b-353d6377de9f" containerName="registry"
Jul 13 23:43:53.049409 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.049384    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.054233 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.054184    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 23:43:53.054233 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.054209    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 23:43:53.054404 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.054212    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 23:43:53.054404 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.054185    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 23:43:53.054404 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.054209    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x2sjv\""
Jul 13 23:43:53.066709 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.066688    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"]
Jul 13 23:43:53.187309 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.187275    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6s48\" (UniqueName: \"kubernetes.io/projected/37d78dee-ed79-4803-8926-4af54d0d5719-kube-api-access-k6s48\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.187309 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.187309    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37d78dee-ed79-4803-8926-4af54d0d5719-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.187514 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.187344    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37d78dee-ed79-4803-8926-4af54d0d5719-manifests\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.187514 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.187364    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37d78dee-ed79-4803-8926-4af54d0d5719-webhook-cert\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.187514 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.187407    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d78dee-ed79-4803-8926-4af54d0d5719-tmp\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.287873 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.287840    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6s48\" (UniqueName: \"kubernetes.io/projected/37d78dee-ed79-4803-8926-4af54d0d5719-kube-api-access-k6s48\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.287873 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.287874    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37d78dee-ed79-4803-8926-4af54d0d5719-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.288100 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.288009    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37d78dee-ed79-4803-8926-4af54d0d5719-manifests\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.288100 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.288045    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37d78dee-ed79-4803-8926-4af54d0d5719-webhook-cert\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.288100 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.288066    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d78dee-ed79-4803-8926-4af54d0d5719-tmp\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.288379 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.288349    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/37d78dee-ed79-4803-8926-4af54d0d5719-manifests\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.290309 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.290277    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d78dee-ed79-4803-8926-4af54d0d5719-tmp\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.290465 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.290446    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/37d78dee-ed79-4803-8926-4af54d0d5719-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.290584 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.290567    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37d78dee-ed79-4803-8926-4af54d0d5719-webhook-cert\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.297213 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.297162    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6s48\" (UniqueName: \"kubernetes.io/projected/37d78dee-ed79-4803-8926-4af54d0d5719-kube-api-access-k6s48\") pod \"opendatahub-operator-controller-manager-74bf4db56c-rmhth\" (UID: \"37d78dee-ed79-4803-8926-4af54d0d5719\") " pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.361025 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.360989    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:53.494042 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:53.494021    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"]
Jul 13 23:43:53.496042 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:43:53.496014    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d78dee_ed79_4803_8926_4af54d0d5719.slice/crio-27a6abd5d36e93bcdd4cebdf93433bfc90add823b487208d25c938aad19ebd0b WatchSource:0}: Error finding container 27a6abd5d36e93bcdd4cebdf93433bfc90add823b487208d25c938aad19ebd0b: Status 404 returned error can't find the container with id 27a6abd5d36e93bcdd4cebdf93433bfc90add823b487208d25c938aad19ebd0b
Jul 13 23:43:54.047585 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:54.047552    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth" event={"ID":"37d78dee-ed79-4803-8926-4af54d0d5719","Type":"ContainerStarted","Data":"27a6abd5d36e93bcdd4cebdf93433bfc90add823b487208d25c938aad19ebd0b"}
Jul 13 23:43:57.062039 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:57.061998    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth" event={"ID":"37d78dee-ed79-4803-8926-4af54d0d5719","Type":"ContainerStarted","Data":"3a73caf99a18495bea65c8d7f01ba276d6092d61ab8eb436e3a33d2d6d2fdf0c"}
Jul 13 23:43:58.067409 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:58.067375    2550 generic.go:356] "Generic (PLEG): container finished" podID="37d78dee-ed79-4803-8926-4af54d0d5719" containerID="3a73caf99a18495bea65c8d7f01ba276d6092d61ab8eb436e3a33d2d6d2fdf0c" exitCode=0
Jul 13 23:43:58.067783 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:58.067465    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth" event={"ID":"37d78dee-ed79-4803-8926-4af54d0d5719","Type":"ContainerDied","Data":"3a73caf99a18495bea65c8d7f01ba276d6092d61ab8eb436e3a33d2d6d2fdf0c"}
Jul 13 23:43:59.073784 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:59.073752    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth" event={"ID":"37d78dee-ed79-4803-8926-4af54d0d5719","Type":"ContainerStarted","Data":"19e110329bd1f2557f86314c985727a1c33acd4d9d240d5338c3b86764291976"}
Jul 13 23:43:59.074182 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:59.073825    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:43:59.095483 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:43:59.095431    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth" podStartSLOduration=1.831025369 podStartE2EDuration="6.095416567s" totalImagesPullingTime="3.227545719s" totalInitContainerRuntime="1.036845479s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:43:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:43:59.094399259 +0000 UTC m=+290.892299957" watchObservedRunningTime="2026-07-13 23:43:59.095416567 +0000 UTC m=+290.893317265"
Jul 13 23:44:08.616760 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:08.616727    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:44:08.619405 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:08.619380    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:44:08.625995 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:08.625965    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:44:08.628882 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:08.628864    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:44:08.629147 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:08.629122    2550 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 23:44:10.081254 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:10.081228    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74bf4db56c-rmhth"
Jul 13 23:44:12.576544 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.576514    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"]
Jul 13 23:44:12.578489 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.578434    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.585190 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.585167    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-d5jbc\""
Jul 13 23:44:12.585270 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.585194    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 23:44:12.585270 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.585212    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 23:44:12.669469 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.669437    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"]
Jul 13 23:44:12.749480 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.749449    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e37dd35-277a-4806-be76-13c59c1493df-tls-certs\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.749638 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.749484    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkm5k\" (UniqueName: \"kubernetes.io/projected/8e37dd35-277a-4806-be76-13c59c1493df-kube-api-access-bkm5k\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.749638 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.749566    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e37dd35-277a-4806-be76-13c59c1493df-tmp\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.850386 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.850305    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e37dd35-277a-4806-be76-13c59c1493df-tls-certs\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.850386 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.850349    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkm5k\" (UniqueName: \"kubernetes.io/projected/8e37dd35-277a-4806-be76-13c59c1493df-kube-api-access-bkm5k\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.850589 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.850423    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e37dd35-277a-4806-be76-13c59c1493df-tmp\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.852665 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.852643    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e37dd35-277a-4806-be76-13c59c1493df-tmp\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.852856 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.852839    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e37dd35-277a-4806-be76-13c59c1493df-tls-certs\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.868688 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.868655    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkm5k\" (UniqueName: \"kubernetes.io/projected/8e37dd35-277a-4806-be76-13c59c1493df-kube-api-access-bkm5k\") pod \"kube-auth-proxy-764dccbb5b-4f2kx\" (UID: \"8e37dd35-277a-4806-be76-13c59c1493df\") " pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:12.887507 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:12.887483    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"
Jul 13 23:44:13.029111 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:13.029079    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx"]
Jul 13 23:44:13.032377 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:44:13.032343    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e37dd35_277a_4806_be76_13c59c1493df.slice/crio-c31aaac114b24e9e5527700bf554f634c2783a6fbbeedf85796a840617071956 WatchSource:0}: Error finding container c31aaac114b24e9e5527700bf554f634c2783a6fbbeedf85796a840617071956: Status 404 returned error can't find the container with id c31aaac114b24e9e5527700bf554f634c2783a6fbbeedf85796a840617071956
Jul 13 23:44:13.034228 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:13.034213    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 23:44:13.131682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:13.131611    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx" event={"ID":"8e37dd35-277a-4806-be76-13c59c1493df","Type":"ContainerStarted","Data":"c31aaac114b24e9e5527700bf554f634c2783a6fbbeedf85796a840617071956"}
Jul 13 23:44:16.148338 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:16.148257    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx" event={"ID":"8e37dd35-277a-4806-be76-13c59c1493df","Type":"ContainerStarted","Data":"26001ce8f5fb44d8faf304d85389c89ec614e7da8359c77c086470f0b4b445f0"}
Jul 13 23:44:16.163109 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:16.163064    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-764dccbb5b-4f2kx" podStartSLOduration=1.4161488740000001 podStartE2EDuration="4.163052531s" totalImagesPullingTime="2.746903657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:44:16.162210343 +0000 UTC m=+307.960111070" watchObservedRunningTime="2026-07-13 23:44:16.163052531 +0000 UTC m=+307.960953229"
Jul 13 23:44:19.530223 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.530192    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"]
Jul 13 23:44:19.532470 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.532454    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.539555 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.539532    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7wzgf\""
Jul 13 23:44:19.539680 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.539580    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 13 23:44:19.551137 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.551110    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"]
Jul 13 23:44:19.608960 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.608934    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609095 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.608965    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39270a84-2b24-471a-a015-b0071a524724-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609095 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.608987    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39270a84-2b24-471a-a015-b0071a524724-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609095 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.609076    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609266 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.609110    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39270a84-2b24-471a-a015-b0071a524724-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609266 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.609182    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609266 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.609215    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609266 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.609241    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.609440 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.609284    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb6sw\" (UniqueName: \"kubernetes.io/projected/39270a84-2b24-471a-a015-b0071a524724-kube-api-access-pb6sw\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.709820 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.709791    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.709820 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.709825    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710059 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.709867    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb6sw\" (UniqueName: \"kubernetes.io/projected/39270a84-2b24-471a-a015-b0071a524724-kube-api-access-pb6sw\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710059 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.709942    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710059 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.709966    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39270a84-2b24-471a-a015-b0071a524724-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710059 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.709995    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39270a84-2b24-471a-a015-b0071a524724-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710142    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710183    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39270a84-2b24-471a-a015-b0071a524724-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710223    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710235    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710468 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710320    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710468 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710448    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710615 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710596    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.710841 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.710821    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39270a84-2b24-471a-a015-b0071a524724-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.712177 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.712155    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39270a84-2b24-471a-a015-b0071a524724-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.712522 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.712502    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39270a84-2b24-471a-a015-b0071a524724-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.723493 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.723468    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39270a84-2b24-471a-a015-b0071a524724-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.723656 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.723638    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb6sw\" (UniqueName: \"kubernetes.io/projected/39270a84-2b24-471a-a015-b0071a524724-kube-api-access-pb6sw\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678l9l96\" (UID: \"39270a84-2b24-471a-a015-b0071a524724\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:19.842542 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:19.842474    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:20.004839 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.004817    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"]
Jul 13 23:44:20.007410 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:44:20.007380    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39270a84_2b24_471a_a015_b0071a524724.slice/crio-9c6ed4648f84415f2b7725439ef7cf9ff18297b2a08f7369d52baad0acc87a2a WatchSource:0}: Error finding container 9c6ed4648f84415f2b7725439ef7cf9ff18297b2a08f7369d52baad0acc87a2a: Status 404 returned error can't find the container with id 9c6ed4648f84415f2b7725439ef7cf9ff18297b2a08f7369d52baad0acc87a2a
Jul 13 23:44:20.165183 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.165104    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96" event={"ID":"39270a84-2b24-471a-a015-b0071a524724","Type":"ContainerStarted","Data":"9c6ed4648f84415f2b7725439ef7cf9ff18297b2a08f7369d52baad0acc87a2a"}
Jul 13 23:44:20.333914 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.333870    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"]
Jul 13 23:44:20.340336 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.340308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.343463 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.343443    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 23:44:20.343463 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.343458    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hbncl\""
Jul 13 23:44:20.343633 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.343502    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 23:44:20.343633 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.343522    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 23:44:20.343709 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.343645    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 23:44:20.343909 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.343877    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 23:44:20.349986 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.349968    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"]
Jul 13 23:44:20.416918 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.416827    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3db368c6-e1dd-4bfa-85d7-6932d54de99d-metrics-cert\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.417029 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.416958    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zndh\" (UniqueName: \"kubernetes.io/projected/3db368c6-e1dd-4bfa-85d7-6932d54de99d-kube-api-access-8zndh\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.417029 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.417003    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3db368c6-e1dd-4bfa-85d7-6932d54de99d-cert\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.417119 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.417028    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3db368c6-e1dd-4bfa-85d7-6932d54de99d-manager-config\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.518162 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.518142    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zndh\" (UniqueName: \"kubernetes.io/projected/3db368c6-e1dd-4bfa-85d7-6932d54de99d-kube-api-access-8zndh\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.518271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.518176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3db368c6-e1dd-4bfa-85d7-6932d54de99d-cert\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.518271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.518194    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3db368c6-e1dd-4bfa-85d7-6932d54de99d-manager-config\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.518271 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.518225    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3db368c6-e1dd-4bfa-85d7-6932d54de99d-metrics-cert\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.518841 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.518819    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3db368c6-e1dd-4bfa-85d7-6932d54de99d-manager-config\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.520605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.520585    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3db368c6-e1dd-4bfa-85d7-6932d54de99d-cert\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.520692 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.520643    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3db368c6-e1dd-4bfa-85d7-6932d54de99d-metrics-cert\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.532711 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.532694    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zndh\" (UniqueName: \"kubernetes.io/projected/3db368c6-e1dd-4bfa-85d7-6932d54de99d-kube-api-access-8zndh\") pod \"lws-controller-manager-5dc78dff7b-7tzdb\" (UID: \"3db368c6-e1dd-4bfa-85d7-6932d54de99d\") " pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.650884 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.650850    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:20.827320 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:20.827139    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"]
Jul 13 23:44:20.974367 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:44:20.974291    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db368c6_e1dd_4bfa_85d7_6932d54de99d.slice/crio-b12cb8b62d90e74215d214948caf8c034c989aa3e2c3412846780612c0621bb0 WatchSource:0}: Error finding container b12cb8b62d90e74215d214948caf8c034c989aa3e2c3412846780612c0621bb0: Status 404 returned error can't find the container with id b12cb8b62d90e74215d214948caf8c034c989aa3e2c3412846780612c0621bb0
Jul 13 23:44:21.177328 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:21.177279    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb" event={"ID":"3db368c6-e1dd-4bfa-85d7-6932d54de99d","Type":"ContainerStarted","Data":"b12cb8b62d90e74215d214948caf8c034c989aa3e2c3412846780612c0621bb0"}
Jul 13 23:44:22.541670 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:22.541635    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 23:44:22.541954 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:22.541703    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 23:44:22.541954 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:22.541730    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 13 23:44:23.189485 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.189449    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96" event={"ID":"39270a84-2b24-471a-a015-b0071a524724","Type":"ContainerStarted","Data":"5c39ee7c7a408f9f2d5b552e2abbb6b5b1e5b3f28faeb14a410dcc7978756303"}
Jul 13 23:44:23.225542 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.225479    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96" podStartSLOduration=1.6934428929999998 podStartE2EDuration="4.225458217s" totalImagesPullingTime="2.532015324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:44:23.224183939 +0000 UTC m=+315.022084637" watchObservedRunningTime="2026-07-13 23:44:23.225458217 +0000 UTC m=+315.023358917"
Jul 13 23:44:23.494245 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.494155    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"]
Jul 13 23:44:23.501354 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.501329    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.504874 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.504689    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pc4lt\""
Jul 13 23:44:23.505772 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.505578    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 13 23:44:23.529113 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.529089    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"]
Jul 13 23:44:23.540420 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.540394    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc0ec160-222f-48b2-8217-c316cc26ed8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-96thl\" (UID: \"dc0ec160-222f-48b2-8217-c316cc26ed8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.540551 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.540471    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9p8\" (UniqueName: \"kubernetes.io/projected/dc0ec160-222f-48b2-8217-c316cc26ed8e-kube-api-access-fh9p8\") pod \"llmisvc-controller-manager-777c6bd7d5-96thl\" (UID: \"dc0ec160-222f-48b2-8217-c316cc26ed8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.641479 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.641449    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9p8\" (UniqueName: \"kubernetes.io/projected/dc0ec160-222f-48b2-8217-c316cc26ed8e-kube-api-access-fh9p8\") pod \"llmisvc-controller-manager-777c6bd7d5-96thl\" (UID: \"dc0ec160-222f-48b2-8217-c316cc26ed8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.641963 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.641547    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc0ec160-222f-48b2-8217-c316cc26ed8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-96thl\" (UID: \"dc0ec160-222f-48b2-8217-c316cc26ed8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.644741 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.644715    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dc0ec160-222f-48b2-8217-c316cc26ed8e-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-96thl\" (UID: \"dc0ec160-222f-48b2-8217-c316cc26ed8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.660352 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.660291    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9p8\" (UniqueName: \"kubernetes.io/projected/dc0ec160-222f-48b2-8217-c316cc26ed8e-kube-api-access-fh9p8\") pod \"llmisvc-controller-manager-777c6bd7d5-96thl\" (UID: \"dc0ec160-222f-48b2-8217-c316cc26ed8e\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.814057 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.814022    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:23.843160 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.843130    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:23.848367 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:23.848343    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:24.195013 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:24.194940    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:24.196007 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:24.195981    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678l9l96"
Jul 13 23:44:24.534384 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:24.534361    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"]
Jul 13 23:44:24.534881 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:44:24.534860    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddc0ec160_222f_48b2_8217_c316cc26ed8e.slice/crio-cf9f1cb7f68cfd33a325c6cd13f5149079207232782a65c1130f22dc44f95800 WatchSource:0}: Error finding container cf9f1cb7f68cfd33a325c6cd13f5149079207232782a65c1130f22dc44f95800: Status 404 returned error can't find the container with id cf9f1cb7f68cfd33a325c6cd13f5149079207232782a65c1130f22dc44f95800
Jul 13 23:44:25.201164 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:25.201125    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb" event={"ID":"3db368c6-e1dd-4bfa-85d7-6932d54de99d","Type":"ContainerStarted","Data":"8632370cdff7fd8fa9bbc1cdbd1907b53e759ae3054bb1fc09896c90e5928e7c"}
Jul 13 23:44:25.201576 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:25.201266    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:25.202375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:25.202351    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl" event={"ID":"dc0ec160-222f-48b2-8217-c316cc26ed8e","Type":"ContainerStarted","Data":"cf9f1cb7f68cfd33a325c6cd13f5149079207232782a65c1130f22dc44f95800"}
Jul 13 23:44:25.230299 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:25.230246    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb" podStartSLOduration=1.751366393 podStartE2EDuration="5.230229385s" totalImagesPullingTime="3.478862992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:44:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:44:25.22907745 +0000 UTC m=+317.026978170" watchObservedRunningTime="2026-07-13 23:44:25.230229385 +0000 UTC m=+317.028130083"
Jul 13 23:44:28.219790 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:28.219697    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl" event={"ID":"dc0ec160-222f-48b2-8217-c316cc26ed8e","Type":"ContainerStarted","Data":"68e5a0624fde458141817261a251a4abfb8abb41ba0b11e36d4049ca00b1bcb7"}
Jul 13 23:44:28.220167 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:28.219793    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:44:28.236200 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:28.236160    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl" podStartSLOduration=1.9632531819999999 podStartE2EDuration="5.236146676s" totalImagesPullingTime="3.272893494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:44:28.235748634 +0000 UTC m=+320.033649333" watchObservedRunningTime="2026-07-13 23:44:28.236146676 +0000 UTC m=+320.034047376"
Jul 13 23:44:36.209662 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:36.209632    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5dc78dff7b-7tzdb"
Jul 13 23:44:59.226549 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:44:59.226517    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-96thl"
Jul 13 23:46:55.777993 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.777949    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:46:55.780365 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.780337    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:55.782815 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.782789    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 23:46:55.783012 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.782944    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 23:46:55.783718 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.783704    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 23:46:55.783767 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.783744    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gbt4b\""
Jul 13 23:46:55.790228 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.790203    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:46:55.822606 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.822563    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:46:55.948716 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.948677    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db071360-e86c-4023-8da2-786494ea079a-config-file\") pod \"limitador-limitador-54d6d988d5-pvwzq\" (UID: \"db071360-e86c-4023-8da2-786494ea079a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:55.948962 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:55.948727    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m242z\" (UniqueName: \"kubernetes.io/projected/db071360-e86c-4023-8da2-786494ea079a-kube-api-access-m242z\") pod \"limitador-limitador-54d6d988d5-pvwzq\" (UID: \"db071360-e86c-4023-8da2-786494ea079a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:56.049782 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.049678    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db071360-e86c-4023-8da2-786494ea079a-config-file\") pod \"limitador-limitador-54d6d988d5-pvwzq\" (UID: \"db071360-e86c-4023-8da2-786494ea079a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:56.049782 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.049725    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m242z\" (UniqueName: \"kubernetes.io/projected/db071360-e86c-4023-8da2-786494ea079a-kube-api-access-m242z\") pod \"limitador-limitador-54d6d988d5-pvwzq\" (UID: \"db071360-e86c-4023-8da2-786494ea079a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:56.050409 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.050383    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db071360-e86c-4023-8da2-786494ea079a-config-file\") pod \"limitador-limitador-54d6d988d5-pvwzq\" (UID: \"db071360-e86c-4023-8da2-786494ea079a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:56.058318 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.058288    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m242z\" (UniqueName: \"kubernetes.io/projected/db071360-e86c-4023-8da2-786494ea079a-kube-api-access-m242z\") pod \"limitador-limitador-54d6d988d5-pvwzq\" (UID: \"db071360-e86c-4023-8da2-786494ea079a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:56.092444 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.092398    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:46:56.239960 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.239925    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:46:56.242932 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:46:56.242881    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb071360_e86c_4023_8da2_786494ea079a.slice/crio-57698c6ccc2a17ec2eb7c16f2d61b9a52a45fd7883f701057a67238480d0e435 WatchSource:0}: Error finding container 57698c6ccc2a17ec2eb7c16f2d61b9a52a45fd7883f701057a67238480d0e435: Status 404 returned error can't find the container with id 57698c6ccc2a17ec2eb7c16f2d61b9a52a45fd7883f701057a67238480d0e435
Jul 13 23:46:56.431531 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.431403    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-vg2vn"]
Jul 13 23:46:56.435958 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.435923    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:46:56.439601 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.439574    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dmwlm\""
Jul 13 23:46:56.441009 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.440985    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vg2vn"]
Jul 13 23:46:56.554196 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.554155    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6bqc\" (UniqueName: \"kubernetes.io/projected/badca4e2-b7fa-4132-b9ef-1d821af3801d-kube-api-access-f6bqc\") pod \"authorino-75d556bd77-vg2vn\" (UID: \"badca4e2-b7fa-4132-b9ef-1d821af3801d\") " pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:46:56.654990 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.654947    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6bqc\" (UniqueName: \"kubernetes.io/projected/badca4e2-b7fa-4132-b9ef-1d821af3801d-kube-api-access-f6bqc\") pod \"authorino-75d556bd77-vg2vn\" (UID: \"badca4e2-b7fa-4132-b9ef-1d821af3801d\") " pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:46:56.664429 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.664396    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6bqc\" (UniqueName: \"kubernetes.io/projected/badca4e2-b7fa-4132-b9ef-1d821af3801d-kube-api-access-f6bqc\") pod \"authorino-75d556bd77-vg2vn\" (UID: \"badca4e2-b7fa-4132-b9ef-1d821af3801d\") " pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:46:56.766539 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.766438    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:46:56.882572 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.882430    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq" event={"ID":"db071360-e86c-4023-8da2-786494ea079a","Type":"ContainerStarted","Data":"57698c6ccc2a17ec2eb7c16f2d61b9a52a45fd7883f701057a67238480d0e435"}
Jul 13 23:46:56.970661 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:56.970495    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vg2vn"]
Jul 13 23:46:57.894801 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:46:57.894687    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vg2vn" event={"ID":"badca4e2-b7fa-4132-b9ef-1d821af3801d","Type":"ContainerStarted","Data":"d24c3d011b8d1c57b4b8d5f00de17ecdc4d9ecff9cdf4439e7b788510f44c86b"}
Jul 13 23:47:00.913129 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:00.913081    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq" event={"ID":"db071360-e86c-4023-8da2-786494ea079a","Type":"ContainerStarted","Data":"9732d766dc86e8d2258ae71f1f203fbe661668676e203307dabca36312a83eaf"}
Jul 13 23:47:00.913647 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:00.913163    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:47:00.914666 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:00.914641    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vg2vn" event={"ID":"badca4e2-b7fa-4132-b9ef-1d821af3801d","Type":"ContainerStarted","Data":"0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a"}
Jul 13 23:47:00.932964 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:00.932881    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq" podStartSLOduration=1.4787632 podStartE2EDuration="5.9328614s" totalImagesPullingTime="4.4540982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:46:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:47:00.930933637 +0000 UTC m=+472.728834374" watchObservedRunningTime="2026-07-13 23:47:00.9328614 +0000 UTC m=+472.730762544"
Jul 13 23:47:00.945380 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:00.945319    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-vg2vn" podStartSLOduration=1.181455476 podStartE2EDuration="4.945300064s" totalImagesPullingTime="3.763844588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:47:00.944383039 +0000 UTC m=+472.742283737" watchObservedRunningTime="2026-07-13 23:47:00.945300064 +0000 UTC m=+472.743200761"
Jul 13 23:47:11.920403 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:11.920370    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"
Jul 13 23:47:29.070410 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.070367    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jzpl2"]
Jul 13 23:47:29.135260 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.135214    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jzpl2"]
Jul 13 23:47:29.135452 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.135322    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:29.259918 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.259867    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdh9h\" (UniqueName: \"kubernetes.io/projected/c8279e9f-0131-443d-aee9-5c349bc2cf41-kube-api-access-wdh9h\") pod \"authorino-5d4b67c446-jzpl2\" (UID: \"c8279e9f-0131-443d-aee9-5c349bc2cf41\") " pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:29.307932 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.307866    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jzpl2"]
Jul 13 23:47:29.308263 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:47:29.308234    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wdh9h], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-jzpl2" podUID="c8279e9f-0131-443d-aee9-5c349bc2cf41"
Jul 13 23:47:29.332230 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.332137    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-87fbff4cc-spvxh"]
Jul 13 23:47:29.349536 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.349503    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-87fbff4cc-spvxh"]
Jul 13 23:47:29.349737 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.349623    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:29.360953 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.360925    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdh9h\" (UniqueName: \"kubernetes.io/projected/c8279e9f-0131-443d-aee9-5c349bc2cf41-kube-api-access-wdh9h\") pod \"authorino-5d4b67c446-jzpl2\" (UID: \"c8279e9f-0131-443d-aee9-5c349bc2cf41\") " pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:29.370634 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.370600    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdh9h\" (UniqueName: \"kubernetes.io/projected/c8279e9f-0131-443d-aee9-5c349bc2cf41-kube-api-access-wdh9h\") pod \"authorino-5d4b67c446-jzpl2\" (UID: \"c8279e9f-0131-443d-aee9-5c349bc2cf41\") " pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:29.438890 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.438846    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-87fbff4cc-spvxh"]
Jul 13 23:47:29.439195 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:47:29.439172    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jznbb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-87fbff4cc-spvxh" podUID="b25bf1f4-91f5-4de8-84be-58d4d4bab0b0"
Jul 13 23:47:29.462395 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.462356    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jznbb\" (UniqueName: \"kubernetes.io/projected/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0-kube-api-access-jznbb\") pod \"authorino-87fbff4cc-spvxh\" (UID: \"b25bf1f4-91f5-4de8-84be-58d4d4bab0b0\") " pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:29.465860 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.465830    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79ff9dd8fc-rdkr9"]
Jul 13 23:47:29.486023 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.485989    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79ff9dd8fc-rdkr9"]
Jul 13 23:47:29.486203 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.486105    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.488435 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.488411    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 13 23:47:29.562919 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.562862    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65dce4a3-7cfb-48d0-a59b-647a597f5138-tls-cert\") pod \"authorino-79ff9dd8fc-rdkr9\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") " pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.562919 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.562927    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6lkl\" (UniqueName: \"kubernetes.io/projected/65dce4a3-7cfb-48d0-a59b-647a597f5138-kube-api-access-v6lkl\") pod \"authorino-79ff9dd8fc-rdkr9\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") " pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.563176 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.562959    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jznbb\" (UniqueName: \"kubernetes.io/projected/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0-kube-api-access-jznbb\") pod \"authorino-87fbff4cc-spvxh\" (UID: \"b25bf1f4-91f5-4de8-84be-58d4d4bab0b0\") " pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:29.571472 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.571449    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jznbb\" (UniqueName: \"kubernetes.io/projected/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0-kube-api-access-jznbb\") pod \"authorino-87fbff4cc-spvxh\" (UID: \"b25bf1f4-91f5-4de8-84be-58d4d4bab0b0\") " pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:29.663742 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.663623    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65dce4a3-7cfb-48d0-a59b-647a597f5138-tls-cert\") pod \"authorino-79ff9dd8fc-rdkr9\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") " pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.663742 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.663681    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6lkl\" (UniqueName: \"kubernetes.io/projected/65dce4a3-7cfb-48d0-a59b-647a597f5138-kube-api-access-v6lkl\") pod \"authorino-79ff9dd8fc-rdkr9\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") " pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.666444 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.666418    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65dce4a3-7cfb-48d0-a59b-647a597f5138-tls-cert\") pod \"authorino-79ff9dd8fc-rdkr9\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") " pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.672399 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.672370    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6lkl\" (UniqueName: \"kubernetes.io/projected/65dce4a3-7cfb-48d0-a59b-647a597f5138-kube-api-access-v6lkl\") pod \"authorino-79ff9dd8fc-rdkr9\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") " pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.796318 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.796274    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:47:29.942006 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:29.941889    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79ff9dd8fc-rdkr9"]
Jul 13 23:47:29.945325 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:47:29.945288    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65dce4a3_7cfb_48d0_a59b_647a597f5138.slice/crio-caf1b481b6ce8097dbec4fb9451c230df460ad894fb411aeec21eebbfa56de32 WatchSource:0}: Error finding container caf1b481b6ce8097dbec4fb9451c230df460ad894fb411aeec21eebbfa56de32: Status 404 returned error can't find the container with id caf1b481b6ce8097dbec4fb9451c230df460ad894fb411aeec21eebbfa56de32
Jul 13 23:47:30.052574 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.052534    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:30.052574 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.052544    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9" event={"ID":"65dce4a3-7cfb-48d0-a59b-647a597f5138","Type":"ContainerStarted","Data":"caf1b481b6ce8097dbec4fb9451c230df460ad894fb411aeec21eebbfa56de32"}
Jul 13 23:47:30.052574 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.052572    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:30.059819 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.059788    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:30.062678 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.062656    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:30.167860 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.167822    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jznbb\" (UniqueName: \"kubernetes.io/projected/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0-kube-api-access-jznbb\") pod \"b25bf1f4-91f5-4de8-84be-58d4d4bab0b0\" (UID: \"b25bf1f4-91f5-4de8-84be-58d4d4bab0b0\") "
Jul 13 23:47:30.168298 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.167939    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdh9h\" (UniqueName: \"kubernetes.io/projected/c8279e9f-0131-443d-aee9-5c349bc2cf41-kube-api-access-wdh9h\") pod \"c8279e9f-0131-443d-aee9-5c349bc2cf41\" (UID: \"c8279e9f-0131-443d-aee9-5c349bc2cf41\") "
Jul 13 23:47:30.170297 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.170265    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0-kube-api-access-jznbb" (OuterVolumeSpecName: "kube-api-access-jznbb") pod "b25bf1f4-91f5-4de8-84be-58d4d4bab0b0" (UID: "b25bf1f4-91f5-4de8-84be-58d4d4bab0b0"). InnerVolumeSpecName "kube-api-access-jznbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:47:30.170388 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.170342    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8279e9f-0131-443d-aee9-5c349bc2cf41-kube-api-access-wdh9h" (OuterVolumeSpecName: "kube-api-access-wdh9h") pod "c8279e9f-0131-443d-aee9-5c349bc2cf41" (UID: "c8279e9f-0131-443d-aee9-5c349bc2cf41"). InnerVolumeSpecName "kube-api-access-wdh9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:47:30.269682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.269578    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jznbb\" (UniqueName: \"kubernetes.io/projected/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0-kube-api-access-jznbb\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:47:30.269682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:30.269615    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdh9h\" (UniqueName: \"kubernetes.io/projected/c8279e9f-0131-443d-aee9-5c349bc2cf41-kube-api-access-wdh9h\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:47:31.059277 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.059168    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jzpl2"
Jul 13 23:47:31.059277 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.059201    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9" event={"ID":"65dce4a3-7cfb-48d0-a59b-647a597f5138","Type":"ContainerStarted","Data":"096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac"}
Jul 13 23:47:31.059521 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.059430    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-87fbff4cc-spvxh"
Jul 13 23:47:31.076826 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.076772    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9" podStartSLOduration=1.604860501 podStartE2EDuration="2.07675413s" totalImagesPullingTime="471.893629ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:47:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:47:31.07493934 +0000 UTC m=+502.872840039" watchObservedRunningTime="2026-07-13 23:47:31.07675413 +0000 UTC m=+502.874654828"
Jul 13 23:47:31.104541 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.104509    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jzpl2"]
Jul 13 23:47:31.111696 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.111666    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vg2vn"]
Jul 13 23:47:31.111885 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.111857    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-vg2vn" podUID="badca4e2-b7fa-4132-b9ef-1d821af3801d" containerName="authorino" containerID="cri-o://0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a" gracePeriod=30
Jul 13 23:47:31.115837 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.115807    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jzpl2"]
Jul 13 23:47:31.142870 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.142825    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-87fbff4cc-spvxh"]
Jul 13 23:47:31.157375 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.157330    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-87fbff4cc-spvxh"]
Jul 13 23:47:31.356598 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.356570    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:47:31.482342 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.482299    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6bqc\" (UniqueName: \"kubernetes.io/projected/badca4e2-b7fa-4132-b9ef-1d821af3801d-kube-api-access-f6bqc\") pod \"badca4e2-b7fa-4132-b9ef-1d821af3801d\" (UID: \"badca4e2-b7fa-4132-b9ef-1d821af3801d\") "
Jul 13 23:47:31.485159 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.485120    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/badca4e2-b7fa-4132-b9ef-1d821af3801d-kube-api-access-f6bqc" (OuterVolumeSpecName: "kube-api-access-f6bqc") pod "badca4e2-b7fa-4132-b9ef-1d821af3801d" (UID: "badca4e2-b7fa-4132-b9ef-1d821af3801d"). InnerVolumeSpecName "kube-api-access-f6bqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:47:31.584187 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:31.584149    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6bqc\" (UniqueName: \"kubernetes.io/projected/badca4e2-b7fa-4132-b9ef-1d821af3801d-kube-api-access-f6bqc\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:47:32.065203 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.065161    2550 generic.go:356] "Generic (PLEG): container finished" podID="badca4e2-b7fa-4132-b9ef-1d821af3801d" containerID="0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a" exitCode=0
Jul 13 23:47:32.065445 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.065210    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vg2vn"
Jul 13 23:47:32.065445 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.065262    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vg2vn" event={"ID":"badca4e2-b7fa-4132-b9ef-1d821af3801d","Type":"ContainerDied","Data":"0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a"}
Jul 13 23:47:32.065445 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.065323    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vg2vn" event={"ID":"badca4e2-b7fa-4132-b9ef-1d821af3801d","Type":"ContainerDied","Data":"d24c3d011b8d1c57b4b8d5f00de17ecdc4d9ecff9cdf4439e7b788510f44c86b"}
Jul 13 23:47:32.065445 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.065354    2550 scope.go:117] "RemoveContainer" containerID="0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a"
Jul 13 23:47:32.074817 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.074796    2550 scope.go:117] "RemoveContainer" containerID="0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a"
Jul 13 23:47:32.075149 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:47:32.075126    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a\": container with ID starting with 0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a not found: ID does not exist" containerID="0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a"
Jul 13 23:47:32.075230 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.075172    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a"} err="failed to get container status \"0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a\": rpc error: code = NotFound desc = could not find container \"0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a\": container with ID starting with 0200a3c4059a06c27781cef3d531a2cee7c72045e0c82a280587e018d46ea07a not found: ID does not exist"
Jul 13 23:47:32.084749 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.084721    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vg2vn"]
Jul 13 23:47:32.088229 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.088206    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vg2vn"]
Jul 13 23:47:32.777149 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.777111    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25bf1f4-91f5-4de8-84be-58d4d4bab0b0" path="/var/lib/kubelet/pods/b25bf1f4-91f5-4de8-84be-58d4d4bab0b0/volumes"
Jul 13 23:47:32.777618 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.777385    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="badca4e2-b7fa-4132-b9ef-1d821af3801d" path="/var/lib/kubelet/pods/badca4e2-b7fa-4132-b9ef-1d821af3801d/volumes"
Jul 13 23:47:32.777677 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:47:32.777663    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8279e9f-0131-443d-aee9-5c349bc2cf41" path="/var/lib/kubelet/pods/c8279e9f-0131-443d-aee9-5c349bc2cf41/volumes"
Jul 13 23:48:12.197039 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:48:12.197006    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:48:26.985353 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:48:26.985314    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:48:33.995394 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:48:33.995354    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:48:49.083739 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:48:49.083691    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:49:04.377588 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:04.377548    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:49:08.648846 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:08.648808    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:49:08.650027 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:08.650004    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:49:08.657675 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:08.657647    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:49:08.659300 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:08.659279    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:49:11.588790 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:11.588743    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:49:16.688686 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:49:16.688647    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:50:22.091174 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.091139    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9bd75549b-tnplk"]
Jul 13 23:50:22.091657 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.091493    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="badca4e2-b7fa-4132-b9ef-1d821af3801d" containerName="authorino"
Jul 13 23:50:22.091657 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.091505    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="badca4e2-b7fa-4132-b9ef-1d821af3801d" containerName="authorino"
Jul 13 23:50:22.091657 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.091570    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="badca4e2-b7fa-4132-b9ef-1d821af3801d" containerName="authorino"
Jul 13 23:50:22.094667 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.094649    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.103755 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.103728    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bd75549b-tnplk"]
Jul 13 23:50:22.256518 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.256478    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0431311a-49af-49d7-8d87-3c7d2df6208d-tls-cert\") pod \"authorino-9bd75549b-tnplk\" (UID: \"0431311a-49af-49d7-8d87-3c7d2df6208d\") " pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.256708 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.256532    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h47js\" (UniqueName: \"kubernetes.io/projected/0431311a-49af-49d7-8d87-3c7d2df6208d-kube-api-access-h47js\") pod \"authorino-9bd75549b-tnplk\" (UID: \"0431311a-49af-49d7-8d87-3c7d2df6208d\") " pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.357698 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.357586    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0431311a-49af-49d7-8d87-3c7d2df6208d-tls-cert\") pod \"authorino-9bd75549b-tnplk\" (UID: \"0431311a-49af-49d7-8d87-3c7d2df6208d\") " pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.357698 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.357652    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h47js\" (UniqueName: \"kubernetes.io/projected/0431311a-49af-49d7-8d87-3c7d2df6208d-kube-api-access-h47js\") pod \"authorino-9bd75549b-tnplk\" (UID: \"0431311a-49af-49d7-8d87-3c7d2df6208d\") " pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.360313 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.360284    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0431311a-49af-49d7-8d87-3c7d2df6208d-tls-cert\") pod \"authorino-9bd75549b-tnplk\" (UID: \"0431311a-49af-49d7-8d87-3c7d2df6208d\") " pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.366074 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.366050    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h47js\" (UniqueName: \"kubernetes.io/projected/0431311a-49af-49d7-8d87-3c7d2df6208d-kube-api-access-h47js\") pod \"authorino-9bd75549b-tnplk\" (UID: \"0431311a-49af-49d7-8d87-3c7d2df6208d\") " pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.405082 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.405030    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9bd75549b-tnplk"
Jul 13 23:50:22.543116 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.543081    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9bd75549b-tnplk"]
Jul 13 23:50:22.547189 ip-10-0-137-167 kubenswrapper[2550]: W0713 23:50:22.547159    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0431311a_49af_49d7_8d87_3c7d2df6208d.slice/crio-d5d971ab1a4a1c7d6257e2cdd9b60618db1078c48a7b409f04553fe647890317 WatchSource:0}: Error finding container d5d971ab1a4a1c7d6257e2cdd9b60618db1078c48a7b409f04553fe647890317: Status 404 returned error can't find the container with id d5d971ab1a4a1c7d6257e2cdd9b60618db1078c48a7b409f04553fe647890317
Jul 13 23:50:22.548587 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.548567    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 23:50:22.850892 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:22.850857    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bd75549b-tnplk" event={"ID":"0431311a-49af-49d7-8d87-3c7d2df6208d","Type":"ContainerStarted","Data":"d5d971ab1a4a1c7d6257e2cdd9b60618db1078c48a7b409f04553fe647890317"}
Jul 13 23:50:23.859731 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:23.859688    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9bd75549b-tnplk" event={"ID":"0431311a-49af-49d7-8d87-3c7d2df6208d","Type":"ContainerStarted","Data":"086544683c633c4d9dd94c30246a5f2a1c8354cf743ebcbebb99973f399c49ed"}
Jul 13 23:50:23.879031 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:23.878968    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9bd75549b-tnplk" podStartSLOduration=1.330106499 podStartE2EDuration="1.878948085s" totalImagesPullingTime="548.841586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 23:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 23:50:23.876794657 +0000 UTC m=+675.674695357" watchObservedRunningTime="2026-07-13 23:50:23.878948085 +0000 UTC m=+675.676848788"
Jul 13 23:50:23.913925 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:23.913612    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79ff9dd8fc-rdkr9"]
Jul 13 23:50:23.913925 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:23.913832    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9" podUID="65dce4a3-7cfb-48d0-a59b-647a597f5138" containerName="authorino" containerID="cri-o://096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac" gracePeriod=30
Jul 13 23:50:24.180911 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.180856    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:50:24.277882 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.277197    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6lkl\" (UniqueName: \"kubernetes.io/projected/65dce4a3-7cfb-48d0-a59b-647a597f5138-kube-api-access-v6lkl\") pod \"65dce4a3-7cfb-48d0-a59b-647a597f5138\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") "
Jul 13 23:50:24.277882 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.277342    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65dce4a3-7cfb-48d0-a59b-647a597f5138-tls-cert\") pod \"65dce4a3-7cfb-48d0-a59b-647a597f5138\" (UID: \"65dce4a3-7cfb-48d0-a59b-647a597f5138\") "
Jul 13 23:50:24.281143 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.281095    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65dce4a3-7cfb-48d0-a59b-647a597f5138-kube-api-access-v6lkl" (OuterVolumeSpecName: "kube-api-access-v6lkl") pod "65dce4a3-7cfb-48d0-a59b-647a597f5138" (UID: "65dce4a3-7cfb-48d0-a59b-647a597f5138"). InnerVolumeSpecName "kube-api-access-v6lkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 23:50:24.292169 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.292130    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65dce4a3-7cfb-48d0-a59b-647a597f5138-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "65dce4a3-7cfb-48d0-a59b-647a597f5138" (UID: "65dce4a3-7cfb-48d0-a59b-647a597f5138"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 23:50:24.378960 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.378919    2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65dce4a3-7cfb-48d0-a59b-647a597f5138-tls-cert\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:50:24.378960 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.378959    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6lkl\" (UniqueName: \"kubernetes.io/projected/65dce4a3-7cfb-48d0-a59b-647a597f5138-kube-api-access-v6lkl\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 13 23:50:24.867287 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.867246    2550 generic.go:356] "Generic (PLEG): container finished" podID="65dce4a3-7cfb-48d0-a59b-647a597f5138" containerID="096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac" exitCode=0
Jul 13 23:50:24.867287 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.867289    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9"
Jul 13 23:50:24.867803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.867319    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9" event={"ID":"65dce4a3-7cfb-48d0-a59b-647a597f5138","Type":"ContainerDied","Data":"096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac"}
Jul 13 23:50:24.867803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.867363    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79ff9dd8fc-rdkr9" event={"ID":"65dce4a3-7cfb-48d0-a59b-647a597f5138","Type":"ContainerDied","Data":"caf1b481b6ce8097dbec4fb9451c230df460ad894fb411aeec21eebbfa56de32"}
Jul 13 23:50:24.867803 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.867381    2550 scope.go:117] "RemoveContainer" containerID="096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac"
Jul 13 23:50:24.878650 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.878623    2550 scope.go:117] "RemoveContainer" containerID="096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac"
Jul 13 23:50:24.879084 ip-10-0-137-167 kubenswrapper[2550]: E0713 23:50:24.879057    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac\": container with ID starting with 096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac not found: ID does not exist" containerID="096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac"
Jul 13 23:50:24.879222 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.879099    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac"} err="failed to get container status \"096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac\": rpc error: code = NotFound desc = could not find container \"096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac\": container with ID starting with 096cc9270c1ff6bf74285661ee3f7c8640584dd58e72087cb6252ccc3962a6ac not found: ID does not exist"
Jul 13 23:50:24.885694 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.885657    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-79ff9dd8fc-rdkr9"]
Jul 13 23:50:24.887690 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:24.887659    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-79ff9dd8fc-rdkr9"]
Jul 13 23:50:26.776226 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:26.776189    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65dce4a3-7cfb-48d0-a59b-647a597f5138" path="/var/lib/kubelet/pods/65dce4a3-7cfb-48d0-a59b-647a597f5138/volumes"
Jul 13 23:50:51.787546 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:50:51.787498    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:51:01.089630 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:51:01.089414    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:51:12.179920 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:51:12.179858    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:51:21.287306 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:51:21.287266    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:51:32.484605 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:51:32.484563    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:51:59.789523 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:51:59.789482    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:52:11.985514 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:52:11.985477    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:52:45.590603 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:52:45.590509    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:52:56.083040 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:52:56.082999    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:53:09.392682 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:53:09.392646    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:53:15.776092 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:53:15.776052    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:53:26.482752 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:53:26.482688    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:53:35.682786 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:53:35.682750    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:53:47.976171 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:53:47.976129    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:53:57.684814 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:53:57.684707    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:54:08.680212 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:08.680183    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:54:08.681013 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:08.680987    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:54:08.691055 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:08.691032    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:54:08.692149 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:08.692127    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:54:22.391264 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:22.391224    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:54:33.183414 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:33.183375    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:54:55.983697 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:54:55.983657    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:55:07.677161 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:55:07.677121    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:55:21.295414 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:55:21.295374    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:55:32.484800 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:55:32.484690    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:55:41.879713 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:55:41.879673    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:55:50.988894 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:55:50.988850    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:56:00.480242 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:56:00.480211    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:57:48.093913 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:57:48.093867    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:57:58.382224 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:57:58.382172    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:58:08.779590 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:58:08.779559    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:58:20.786909 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:58:20.786855    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:58:30.179461 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:58:30.179372    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:58:41.183547 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:58:41.183507    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:58:49.984033 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:58:49.983990    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:59:00.888607 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:00.888566    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:59:08.710770 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:08.710742    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:59:08.711733 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:08.711704    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 13 23:59:08.719299 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:08.719279    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:59:08.720001 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:08.719979    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 13 23:59:12.278725 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:12.278674    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:59:20.777640 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:20.777608    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:59:32.087325 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:32.087282    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 13 23:59:53.183855 ip-10-0-137-167 kubenswrapper[2550]: I0713 23:59:53.183813    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:00:02.683011 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:00:02.682878    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:00:12.488869 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:00:12.488824    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:00:23.183850 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:00:23.183804    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:00:41.281481 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:00:41.281441    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:00:51.084705 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:00:51.084665    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:00:59.980272 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:00:59.980234    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:01:09.979220 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:01:09.979180    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:01:18.880880 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:01:18.880839    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:01:27.876700 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:01:27.876659    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:01:41.679733 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:01:41.679653    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:01:50.986372 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:01:50.986332    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:02:00.580785 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:02:00.580749    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:02:13.174888 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:02:13.174852    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:02:22.590548 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:02:22.590510    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:02:29.476683 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:02:29.476642    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:02:45.978000 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:02:45.977952    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:02:55.634628 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:02:55.634594    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:04.982514 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:04.982433    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:16.687865 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:16.687825    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:27.478335 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:27.478300    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:36.180216 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:36.180184    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:45.117152 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:45.117115    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:54.979585 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:54.979549    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:03:55.858973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.858935    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"]
Jul 14 00:03:55.859345 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.859328    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65dce4a3-7cfb-48d0-a59b-647a597f5138" containerName="authorino"
Jul 14 00:03:55.859438 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.859348    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="65dce4a3-7cfb-48d0-a59b-647a597f5138" containerName="authorino"
Jul 14 00:03:55.859490 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.859445    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="65dce4a3-7cfb-48d0-a59b-647a597f5138" containerName="authorino"
Jul 14 00:03:55.862537 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.862516    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:03:55.864999 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.864976    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 00:03:55.865092 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.865041    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vzxsj\""
Jul 14 00:03:55.871214 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:55.871135    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"]
Jul 14 00:03:56.028286 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.028241    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca727a49-5e41-4a46-882f-7a5b2d33497c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30780878-4m89b\" (UID: \"ca727a49-5e41-4a46-882f-7a5b2d33497c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:03:56.129124 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.129046    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca727a49-5e41-4a46-882f-7a5b2d33497c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30780878-4m89b\" (UID: \"ca727a49-5e41-4a46-882f-7a5b2d33497c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:03:56.129617 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.129598    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca727a49-5e41-4a46-882f-7a5b2d33497c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-30780878-4m89b\" (UID: \"ca727a49-5e41-4a46-882f-7a5b2d33497c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:03:56.179664 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.179628    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:03:56.313941 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.313884    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"]
Jul 14 00:03:56.316534 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:03:56.316507    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca727a49_5e41_4a46_882f_7a5b2d33497c.slice/crio-50cc6414d1bdbd384082248add754c924f840105fbabd63ea6ead94f13a205f5 WatchSource:0}: Error finding container 50cc6414d1bdbd384082248add754c924f840105fbabd63ea6ead94f13a205f5: Status 404 returned error can't find the container with id 50cc6414d1bdbd384082248add754c924f840105fbabd63ea6ead94f13a205f5
Jul 14 00:03:56.318535 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.318515    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 00:03:56.648734 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:03:56.648698    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b" event={"ID":"ca727a49-5e41-4a46-882f-7a5b2d33497c","Type":"ContainerStarted","Data":"50cc6414d1bdbd384082248add754c924f840105fbabd63ea6ead94f13a205f5"}
Jul 14 00:04:00.667912 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:00.667810    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b" event={"ID":"ca727a49-5e41-4a46-882f-7a5b2d33497c","Type":"ContainerStarted","Data":"f6ec7124104f56888365fd8057b7990542908ddfc0e6b222d7eba8a63168b9f4"}
Jul 14 00:04:00.684824 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:00.684779    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b" podStartSLOduration=2.041054105 podStartE2EDuration="5.684763293s" totalImagesPullingTime="3.643709188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:04:00.682328006 +0000 UTC m=+1492.480228703" watchObservedRunningTime="2026-07-14 00:04:00.684763293 +0000 UTC m=+1492.482663990"
Jul 14 00:04:01.674023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:01.673991    2550 generic.go:356] "Generic (PLEG): container finished" podID="ca727a49-5e41-4a46-882f-7a5b2d33497c" containerID="f6ec7124104f56888365fd8057b7990542908ddfc0e6b222d7eba8a63168b9f4" exitCode=0
Jul 14 00:04:01.674389 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:01.674088    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b" event={"ID":"ca727a49-5e41-4a46-882f-7a5b2d33497c","Type":"ContainerDied","Data":"f6ec7124104f56888365fd8057b7990542908ddfc0e6b222d7eba8a63168b9f4"}
Jul 14 00:04:02.805665 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:02.805639    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:04:02.992212 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:02.992130    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca727a49-5e41-4a46-882f-7a5b2d33497c-maas-api-service-ca\") pod \"ca727a49-5e41-4a46-882f-7a5b2d33497c\" (UID: \"ca727a49-5e41-4a46-882f-7a5b2d33497c\") "
Jul 14 00:04:02.992475 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:02.992451    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca727a49-5e41-4a46-882f-7a5b2d33497c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca727a49-5e41-4a46-882f-7a5b2d33497c" (UID: "ca727a49-5e41-4a46-882f-7a5b2d33497c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:04:03.093438 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:03.093399    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca727a49-5e41-4a46-882f-7a5b2d33497c-maas-api-service-ca\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:04:03.683918 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:03.683867    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b" event={"ID":"ca727a49-5e41-4a46-882f-7a5b2d33497c","Type":"ContainerDied","Data":"50cc6414d1bdbd384082248add754c924f840105fbabd63ea6ead94f13a205f5"}
Jul 14 00:04:03.684081 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:03.683939    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50cc6414d1bdbd384082248add754c924f840105fbabd63ea6ead94f13a205f5"
Jul 14 00:04:03.684081 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:03.683888    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-30780878-4m89b"
Jul 14 00:04:08.739936 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:08.739890    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:04:08.741306 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:08.741281    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:04:08.751025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:08.750999    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:04:08.752280 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:08.752260    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:04:11.272955 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:11.272908    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:04:21.649672 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.649632    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"]
Jul 14 00:04:21.650218 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.650199    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca727a49-5e41-4a46-882f-7a5b2d33497c" containerName="revoke-keys"
Jul 14 00:04:21.650266 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.650221    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca727a49-5e41-4a46-882f-7a5b2d33497c" containerName="revoke-keys"
Jul 14 00:04:21.650329 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.650317    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca727a49-5e41-4a46-882f-7a5b2d33497c" containerName="revoke-keys"
Jul 14 00:04:21.658350 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.658326    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.660947 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.660917    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-982b7aec-gw-openshift-default-dockercfg-gflhz\""
Jul 14 00:04:21.665428 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.665395    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"]
Jul 14 00:04:21.755989 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.755952    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-credential-socket\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.755989 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.755992    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-data\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756242 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756020    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-istio-token\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756242 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756065    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-certs\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756242 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756114    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvvp9\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-kube-api-access-nvvp9\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756242 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756166    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd768842-49bd-4c4b-b427-4e7109290682-istio-podinfo\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756242 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756236    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd768842-49bd-4c4b-b427-4e7109290682-istiod-ca-cert\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756416 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756255    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-envoy\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.756416 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.756316    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-socket\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857085 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857038    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-credential-socket\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857085 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857080    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-data\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857098    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-istio-token\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857120    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-certs\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857154    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvvp9\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-kube-api-access-nvvp9\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857175    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd768842-49bd-4c4b-b427-4e7109290682-istio-podinfo\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857215    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd768842-49bd-4c4b-b427-4e7109290682-istiod-ca-cert\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857242    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-envoy\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857643 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857437    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-socket\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857643 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857519    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-data\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857643 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857612    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-certs\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857793 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857738    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-credential-socket\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.857885 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.857862    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-socket\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.858103 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.858085    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd768842-49bd-4c4b-b427-4e7109290682-istiod-ca-cert\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.859771 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.859750    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-envoy\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.860040 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.860019    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd768842-49bd-4c4b-b427-4e7109290682-istio-podinfo\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.865656 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.865630    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-istio-token\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.866194 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.866172    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvvp9\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-kube-api-access-nvvp9\") pod \"e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") " pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:21.970333 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:21.970232    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:22.118405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.118376    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"]
Jul 14 00:04:22.121311 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:04:22.121275    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd768842_49bd_4c4b_b427_4e7109290682.slice/crio-e3f996cc6e439154a3e1c07032f1a6381d24a7c923e17335a7bcae81a61a209e WatchSource:0}: Error finding container e3f996cc6e439154a3e1c07032f1a6381d24a7c923e17335a7bcae81a61a209e: Status 404 returned error can't find the container with id e3f996cc6e439154a3e1c07032f1a6381d24a7c923e17335a7bcae81a61a209e
Jul 14 00:04:22.123938 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.123885    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:04:22.124023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.123972    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:04:22.124023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.124000    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:04:22.776717 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.776678    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" event={"ID":"dd768842-49bd-4c4b-b427-4e7109290682","Type":"ContainerStarted","Data":"e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"}
Jul 14 00:04:22.777114 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.776736    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" event={"ID":"dd768842-49bd-4c4b-b427-4e7109290682","Type":"ContainerStarted","Data":"e3f996cc6e439154a3e1c07032f1a6381d24a7c923e17335a7bcae81a61a209e"}
Jul 14 00:04:22.792509 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.792433    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podStartSLOduration=1.7924129629999999 podStartE2EDuration="1.792412963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:04:22.791874693 +0000 UTC m=+1514.589775405" watchObservedRunningTime="2026-07-14 00:04:22.792412963 +0000 UTC m=+1514.590313662"
Jul 14 00:04:22.971084 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:22.971047    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:23.975396 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:23.975362    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:24.781613 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:24.781579    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:24.782526 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:24.782508    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:25.076063 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:25.075966    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:04:33.831288 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:33.831250    2550 generic.go:356] "Generic (PLEG): container finished" podID="dd768842-49bd-4c4b-b427-4e7109290682" containerID="e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715" exitCode=0
Jul 14 00:04:33.831678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:33.831319    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" event={"ID":"dd768842-49bd-4c4b-b427-4e7109290682","Type":"ContainerDied","Data":"e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"}
Jul 14 00:04:33.831751 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:33.831735    2550 scope.go:117] "RemoveContainer" containerID="e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"
Jul 14 00:04:33.832811 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:33.832785    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:04:33.832888 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:33.832871    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:04:33.832968 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:33.832952    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:04:34.837917 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:34.837864    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" event={"ID":"dd768842-49bd-4c4b-b427-4e7109290682","Type":"ContainerStarted","Data":"eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0"}
Jul 14 00:04:34.970946 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:34.970840    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:34.971815 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:34.971779    2550 patch_prober.go:28] interesting pod/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 14 00:04:34.971990 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:34.971847    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 14 00:04:35.971305 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:35.971267    2550 patch_prober.go:28] interesting pod/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 14 00:04:35.971694 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:35.971328    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 14 00:04:36.971230 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:36.971189    2550 patch_prober.go:28] interesting pod/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 14 00:04:36.971418 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:36.971252    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 14 00:04:37.971601 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:37.971566    2550 patch_prober.go:28] interesting pod/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 14 00:04:37.972024 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:37.971625    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 14 00:04:38.970726 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:38.970684    2550 patch_prober.go:28] interesting pod/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 14 00:04:38.970951 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:38.970740    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 14 00:04:39.782344 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:39.782302    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:39.975384 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:39.975351    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:04:39.976469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:04:39.976448    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:05:27.538038 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:27.538003    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"]
Jul 14 00:05:27.538595 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:27.538263    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy" containerID="cri-o://eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0" gracePeriod=30
Jul 14 00:05:30.482018 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:30.481973    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:05:32.813558 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.813527    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:05:32.904193 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904142    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-certs\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904370 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904216    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-istio-token\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904370 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904250    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-socket\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904370 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904299    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd768842-49bd-4c4b-b427-4e7109290682-istiod-ca-cert\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904370 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904326    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-credential-socket\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904370 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904350    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd768842-49bd-4c4b-b427-4e7109290682-istio-podinfo\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904711 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904384    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-envoy\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904711 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904460    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-data\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904711 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904485    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvvp9\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-kube-api-access-nvvp9\") pod \"dd768842-49bd-4c4b-b427-4e7109290682\" (UID: \"dd768842-49bd-4c4b-b427-4e7109290682\") "
Jul 14 00:05:32.904940 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.904893    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:05:32.907284 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.907237    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd768842-49bd-4c4b-b427-4e7109290682-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:05:32.907493 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.907464    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:05:32.908206 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.908085    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:05:32.908206 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.908102    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:05:32.909981 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.908541    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-kube-api-access-nvvp9" (OuterVolumeSpecName: "kube-api-access-nvvp9") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "kube-api-access-nvvp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:05:32.909981 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.908641    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:05:32.911946 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.910587    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:05:32.919297 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:32.919236    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd768842-49bd-4c4b-b427-4e7109290682-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd768842-49bd-4c4b-b427-4e7109290682" (UID: "dd768842-49bd-4c4b-b427-4e7109290682"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 00:05:33.005678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005631    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-data\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.005678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005670    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvvp9\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-kube-api-access-nvvp9\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.005678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005681    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.005678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005689    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd768842-49bd-4c4b-b427-4e7109290682-istio-token\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.006004 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005697    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-workload-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.006004 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005706    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd768842-49bd-4c4b-b427-4e7109290682-istiod-ca-cert\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.006004 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005714    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-credential-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.006004 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005722    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd768842-49bd-4c4b-b427-4e7109290682-istio-podinfo\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.006004 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.005730    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd768842-49bd-4c4b-b427-4e7109290682-istio-envoy\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:05:33.111304 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.111205    2550 generic.go:356] "Generic (PLEG): container finished" podID="dd768842-49bd-4c4b-b427-4e7109290682" containerID="eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0" exitCode=0
Jul 14 00:05:33.111304 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.111249    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" event={"ID":"dd768842-49bd-4c4b-b427-4e7109290682","Type":"ContainerDied","Data":"eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0"}
Jul 14 00:05:33.111304 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.111295    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"
Jul 14 00:05:33.111542 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.111338    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz" event={"ID":"dd768842-49bd-4c4b-b427-4e7109290682","Type":"ContainerDied","Data":"e3f996cc6e439154a3e1c07032f1a6381d24a7c923e17335a7bcae81a61a209e"}
Jul 14 00:05:33.111542 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.111370    2550 scope.go:117] "RemoveContainer" containerID="eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0"
Jul 14 00:05:33.120828 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.120810    2550 scope.go:117] "RemoveContainer" containerID="e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"
Jul 14 00:05:33.129605 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.129582    2550 scope.go:117] "RemoveContainer" containerID="eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0"
Jul 14 00:05:33.130026 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:05:33.129991    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0\": container with ID starting with eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0 not found: ID does not exist" containerID="eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0"
Jul 14 00:05:33.130145 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.130034    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0"} err="failed to get container status \"eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0\": rpc error: code = NotFound desc = could not find container \"eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0\": container with ID starting with eb9f9e3f195215b597f81859e8d83678b57641b478df37872b16c30ec76d6ea0 not found: ID does not exist"
Jul 14 00:05:33.130145 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.130060    2550 scope.go:117] "RemoveContainer" containerID="e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"
Jul 14 00:05:33.130671 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:05:33.130645    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715\": container with ID starting with e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715 not found: ID does not exist" containerID="e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"
Jul 14 00:05:33.130735 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.130681    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715"} err="failed to get container status \"e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715\": rpc error: code = NotFound desc = could not find container \"e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715\": container with ID starting with e6cc71d35d44924216929fd90535b12370485e8cfc857f068593208c146ae715 not found: ID does not exist"
Jul 14 00:05:33.132662 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.132642    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"]
Jul 14 00:05:33.138380 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:33.138353    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-982b7aec-gw-openshift-default-7745474467-q69jz"]
Jul 14 00:05:34.777174 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:34.777137    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd768842-49bd-4c4b-b427-4e7109290682" path="/var/lib/kubelet/pods/dd768842-49bd-4c4b-b427-4e7109290682/volumes"
Jul 14 00:05:43.183364 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:43.183323    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:05:45.687105 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687065    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"]
Jul 14 00:05:45.687516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687413    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy"
Jul 14 00:05:45.687516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687424    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy"
Jul 14 00:05:45.687516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687435    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy"
Jul 14 00:05:45.687516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687441    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy"
Jul 14 00:05:45.687516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687501    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy"
Jul 14 00:05:45.687516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.687509    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd768842-49bd-4c4b-b427-4e7109290682" containerName="istio-proxy"
Jul 14 00:05:45.689408 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.689384    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:05:45.691880 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.691856    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 00:05:45.692038 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.691889    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vzxsj\""
Jul 14 00:05:45.698630 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.698601    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"]
Jul 14 00:05:45.723913 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.723861    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8757b4d-2393-4837-a36c-f299ef51c561-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0bcce182-84bgs\" (UID: \"e8757b4d-2393-4837-a36c-f299ef51c561\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:05:45.824553 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.824511    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8757b4d-2393-4837-a36c-f299ef51c561-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0bcce182-84bgs\" (UID: \"e8757b4d-2393-4837-a36c-f299ef51c561\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:05:45.825206 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:45.825173    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8757b4d-2393-4837-a36c-f299ef51c561-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0bcce182-84bgs\" (UID: \"e8757b4d-2393-4837-a36c-f299ef51c561\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:05:46.001413 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:46.001302    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:05:46.142649 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:46.142621    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"]
Jul 14 00:05:46.144738 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:05:46.144709    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8757b4d_2393_4837_a36c_f299ef51c561.slice/crio-0584b6d72bb7ab1477eba597415a89d02dd0e029e15e196dd314a672a89f8616 WatchSource:0}: Error finding container 0584b6d72bb7ab1477eba597415a89d02dd0e029e15e196dd314a672a89f8616: Status 404 returned error can't find the container with id 0584b6d72bb7ab1477eba597415a89d02dd0e029e15e196dd314a672a89f8616
Jul 14 00:05:46.175497 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:46.175464    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" event={"ID":"e8757b4d-2393-4837-a36c-f299ef51c561","Type":"ContainerStarted","Data":"0584b6d72bb7ab1477eba597415a89d02dd0e029e15e196dd314a672a89f8616"}
Jul 14 00:05:47.183518 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:47.183417    2550 generic.go:356] "Generic (PLEG): container finished" podID="e8757b4d-2393-4837-a36c-f299ef51c561" containerID="49cf6ac4776e1d8c0d99430590e43d42451b0be30d066c15afa12b0c23aed45f" exitCode=7
Jul 14 00:05:47.183518 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:47.183463    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" event={"ID":"e8757b4d-2393-4837-a36c-f299ef51c561","Type":"ContainerDied","Data":"49cf6ac4776e1d8c0d99430590e43d42451b0be30d066c15afa12b0c23aed45f"}
Jul 14 00:05:47.183938 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:47.183722    2550 scope.go:117] "RemoveContainer" containerID="49cf6ac4776e1d8c0d99430590e43d42451b0be30d066c15afa12b0c23aed45f"
Jul 14 00:05:48.190810 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:48.190707    2550 generic.go:356] "Generic (PLEG): container finished" podID="e8757b4d-2393-4837-a36c-f299ef51c561" containerID="9a00f7aaf41f90d3a79a39375eb2d8ef843c88f7aa084b07ffcb92afd81f586e" exitCode=7
Jul 14 00:05:48.190810 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:48.190759    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" event={"ID":"e8757b4d-2393-4837-a36c-f299ef51c561","Type":"ContainerDied","Data":"9a00f7aaf41f90d3a79a39375eb2d8ef843c88f7aa084b07ffcb92afd81f586e"}
Jul 14 00:05:48.191291 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:48.190813    2550 scope.go:117] "RemoveContainer" containerID="49cf6ac4776e1d8c0d99430590e43d42451b0be30d066c15afa12b0c23aed45f"
Jul 14 00:05:48.191291 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:48.191015    2550 scope.go:117] "RemoveContainer" containerID="9a00f7aaf41f90d3a79a39375eb2d8ef843c88f7aa084b07ffcb92afd81f586e"
Jul 14 00:05:48.191291 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:05:48.191265    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0bcce182-84bgs_odh-ai-gateway-infra(e8757b4d-2393-4837-a36c-f299ef51c561)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" podUID="e8757b4d-2393-4837-a36c-f299ef51c561"
Jul 14 00:05:49.197704 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:05:49.197675    2550 scope.go:117] "RemoveContainer" containerID="9a00f7aaf41f90d3a79a39375eb2d8ef843c88f7aa084b07ffcb92afd81f586e"
Jul 14 00:05:49.198142 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:05:49.197857    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-0bcce182-84bgs_odh-ai-gateway-infra(e8757b4d-2393-4837-a36c-f299ef51c561)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" podUID="e8757b4d-2393-4837-a36c-f299ef51c561"
Jul 14 00:06:02.771239 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:02.771198    2550 scope.go:117] "RemoveContainer" containerID="9a00f7aaf41f90d3a79a39375eb2d8ef843c88f7aa084b07ffcb92afd81f586e"
Jul 14 00:06:03.269233 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:03.269197    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" event={"ID":"e8757b4d-2393-4837-a36c-f299ef51c561","Type":"ContainerStarted","Data":"bf9d6117d3aae98e0ad653b3bea6dcf94773713850104bf9e1ce1423f0aa21df"}
Jul 14 00:06:03.287368 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:03.287311    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" podStartSLOduration=18.287293611 podStartE2EDuration="18.287293611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:05:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:06:03.285706058 +0000 UTC m=+1615.083606773" watchObservedRunningTime="2026-07-14 00:06:03.287293611 +0000 UTC m=+1615.085194350"
Jul 14 00:06:04.275563 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:04.275526    2550 generic.go:356] "Generic (PLEG): container finished" podID="e8757b4d-2393-4837-a36c-f299ef51c561" containerID="bf9d6117d3aae98e0ad653b3bea6dcf94773713850104bf9e1ce1423f0aa21df" exitCode=0
Jul 14 00:06:04.276073 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:04.275605    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" event={"ID":"e8757b4d-2393-4837-a36c-f299ef51c561","Type":"ContainerDied","Data":"bf9d6117d3aae98e0ad653b3bea6dcf94773713850104bf9e1ce1423f0aa21df"}
Jul 14 00:06:04.276073 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:04.275688    2550 scope.go:117] "RemoveContainer" containerID="9a00f7aaf41f90d3a79a39375eb2d8ef843c88f7aa084b07ffcb92afd81f586e"
Jul 14 00:06:04.297093 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:04.297057    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"]
Jul 14 00:06:05.420412 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:05.420381    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:06:05.507196 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:05.507154    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8757b4d-2393-4837-a36c-f299ef51c561-maas-api-service-ca\") pod \"e8757b4d-2393-4837-a36c-f299ef51c561\" (UID: \"e8757b4d-2393-4837-a36c-f299ef51c561\") "
Jul 14 00:06:05.507534 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:05.507511    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8757b4d-2393-4837-a36c-f299ef51c561-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8757b4d-2393-4837-a36c-f299ef51c561" (UID: "e8757b4d-2393-4837-a36c-f299ef51c561"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:06:05.607860 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:05.607765    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8757b4d-2393-4837-a36c-f299ef51c561-maas-api-service-ca\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:06:06.287517 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:06.287483    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs" event={"ID":"e8757b4d-2393-4837-a36c-f299ef51c561","Type":"ContainerDied","Data":"0584b6d72bb7ab1477eba597415a89d02dd0e029e15e196dd314a672a89f8616"}
Jul 14 00:06:06.287696 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:06.287531    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0584b6d72bb7ab1477eba597415a89d02dd0e029e15e196dd314a672a89f8616"
Jul 14 00:06:06.287696 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:06.287492    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"
Jul 14 00:06:06.315235 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:06.315202    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"]
Jul 14 00:06:06.321465 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:06.321425    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-84bgs"]
Jul 14 00:06:06.776553 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:06.776459    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" path="/var/lib/kubelet/pods/e8757b4d-2393-4837-a36c-f299ef51c561/volumes"
Jul 14 00:06:45.739887 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.739850    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"]
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740231    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740245    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740265    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740274    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740342    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740353    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.740390 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.740361    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:06:45.742258 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.742241    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:45.744526 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.744491    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 00:06:45.744526 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.744505    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vzxsj\""
Jul 14 00:06:45.751291 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.751265    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"]
Jul 14 00:06:45.865739 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.865697    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb4508d-ec32-4735-94d8-8e8811a11dfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2\" (UID: \"7bb4508d-ec32-4735-94d8-8e8811a11dfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:45.966196 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.966150    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb4508d-ec32-4735-94d8-8e8811a11dfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2\" (UID: \"7bb4508d-ec32-4735-94d8-8e8811a11dfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:45.966856 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:45.966836    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb4508d-ec32-4735-94d8-8e8811a11dfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2\" (UID: \"7bb4508d-ec32-4735-94d8-8e8811a11dfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:46.054428 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:46.054386    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:46.194405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:46.194375    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"]
Jul 14 00:06:46.196728 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:06:46.196693    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb4508d_ec32_4735_94d8_8e8811a11dfe.slice/crio-8eb3e2a0e930602f7abe91dbbffa8c165ff6a216422e6cb0c7760906daa0e282 WatchSource:0}: Error finding container 8eb3e2a0e930602f7abe91dbbffa8c165ff6a216422e6cb0c7760906daa0e282: Status 404 returned error can't find the container with id 8eb3e2a0e930602f7abe91dbbffa8c165ff6a216422e6cb0c7760906daa0e282
Jul 14 00:06:46.473655 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:46.473618    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2" event={"ID":"7bb4508d-ec32-4735-94d8-8e8811a11dfe","Type":"ContainerStarted","Data":"74f246620c7b898adef1dbc075ef0f603cf1387d58b88d27baed2293a9f05917"}
Jul 14 00:06:46.473851 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:46.473673    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2" event={"ID":"7bb4508d-ec32-4735-94d8-8e8811a11dfe","Type":"ContainerStarted","Data":"8eb3e2a0e930602f7abe91dbbffa8c165ff6a216422e6cb0c7760906daa0e282"}
Jul 14 00:06:46.489719 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:46.489662    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2" podStartSLOduration=1.4896393190000001 podStartE2EDuration="1.489639319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:06:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:06:46.488850214 +0000 UTC m=+1658.286750914" watchObservedRunningTime="2026-07-14 00:06:46.489639319 +0000 UTC m=+1658.287540017"
Jul 14 00:06:47.484437 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:47.484396    2550 generic.go:356] "Generic (PLEG): container finished" podID="7bb4508d-ec32-4735-94d8-8e8811a11dfe" containerID="74f246620c7b898adef1dbc075ef0f603cf1387d58b88d27baed2293a9f05917" exitCode=0
Jul 14 00:06:47.484852 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:47.484487    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2" event={"ID":"7bb4508d-ec32-4735-94d8-8e8811a11dfe","Type":"ContainerDied","Data":"74f246620c7b898adef1dbc075ef0f603cf1387d58b88d27baed2293a9f05917"}
Jul 14 00:06:48.631639 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:48.631611    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:48.690787 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:48.690751    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb4508d-ec32-4735-94d8-8e8811a11dfe-maas-api-service-ca\") pod \"7bb4508d-ec32-4735-94d8-8e8811a11dfe\" (UID: \"7bb4508d-ec32-4735-94d8-8e8811a11dfe\") "
Jul 14 00:06:48.691180 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:48.691155    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb4508d-ec32-4735-94d8-8e8811a11dfe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7bb4508d-ec32-4735-94d8-8e8811a11dfe" (UID: "7bb4508d-ec32-4735-94d8-8e8811a11dfe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:06:48.792161 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:48.792131    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7bb4508d-ec32-4735-94d8-8e8811a11dfe-maas-api-service-ca\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:06:49.495116 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:49.495083    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2"
Jul 14 00:06:49.495116 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:49.495104    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2" event={"ID":"7bb4508d-ec32-4735-94d8-8e8811a11dfe","Type":"ContainerDied","Data":"8eb3e2a0e930602f7abe91dbbffa8c165ff6a216422e6cb0c7760906daa0e282"}
Jul 14 00:06:49.495388 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:49.495150    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8eb3e2a0e930602f7abe91dbbffa8c165ff6a216422e6cb0c7760906daa0e282"
Jul 14 00:06:59.482928 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:06:59.482879    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:07:12.127274 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.127234    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"]
Jul 14 00:07:12.127660 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.127615    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:07:12.127660 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.127628    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8757b4d-2393-4837-a36c-f299ef51c561" containerName="revoke-keys"
Jul 14 00:07:12.127660 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.127650    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb4508d-ec32-4735-94d8-8e8811a11dfe" containerName="revoke-keys"
Jul 14 00:07:12.127660 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.127656    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb4508d-ec32-4735-94d8-8e8811a11dfe" containerName="revoke-keys"
Jul 14 00:07:12.127790 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.127710    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb4508d-ec32-4735-94d8-8e8811a11dfe" containerName="revoke-keys"
Jul 14 00:07:12.129799 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.129774    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.132541 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.132513    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bb015d31-openshift-default-dockercfg-gmp65\""
Jul 14 00:07:12.141406 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.141380    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"]
Jul 14 00:07:12.210023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.209976    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-socket\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210022    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30079938-c46c-4c38-a6e4-a15bea355fa0-istiod-ca-cert\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210269 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210094    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-credential-socket\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210269 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210146    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d28r\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-kube-api-access-9d28r\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210269 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210169    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-podinfo\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210269 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210241    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-certs\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210425 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210278    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-envoy\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210425 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210337    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-data\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.210425 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.210353    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-token\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311160 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311110    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-podinfo\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311160 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311164    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-certs\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311439 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311276    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-envoy\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311439 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311344    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-data\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311439 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311361    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-token\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311439 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311406    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-socket\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311439 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311421    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30079938-c46c-4c38-a6e4-a15bea355fa0-istiod-ca-cert\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311700 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311470    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-credential-socket\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311700 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311532    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d28r\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-kube-api-access-9d28r\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.311796 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311772    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-data\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.312020 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.311968    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-socket\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.312020 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.312030    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-credential-socket\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.312430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.312402    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-certs\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.312523 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.312453    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30079938-c46c-4c38-a6e4-a15bea355fa0-istiod-ca-cert\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.313831 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.313803    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-envoy\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.314066 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.314048    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-podinfo\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.322254 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.322230    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-token\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.322401 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.322279    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d28r\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-kube-api-access-9d28r\") pod \"e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") " pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.442814 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.442705    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:12.586965 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.586928    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"]
Jul 14 00:07:12.588985 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:07:12.588949    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30079938_c46c_4c38_a6e4_a15bea355fa0.slice/crio-c2073da34e8dce93eb51f2acea1ad730e91855389da3a18f2852137bd4792bc1 WatchSource:0}: Error finding container c2073da34e8dce93eb51f2acea1ad730e91855389da3a18f2852137bd4792bc1: Status 404 returned error can't find the container with id c2073da34e8dce93eb51f2acea1ad730e91855389da3a18f2852137bd4792bc1
Jul 14 00:07:12.591260 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.591227    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:07:12.591362 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.591302    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:07:12.591362 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.591335    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:07:12.609767 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:12.609729    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" event={"ID":"30079938-c46c-4c38-a6e4-a15bea355fa0","Type":"ContainerStarted","Data":"c2073da34e8dce93eb51f2acea1ad730e91855389da3a18f2852137bd4792bc1"}
Jul 14 00:07:13.616749 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:13.616697    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" event={"ID":"30079938-c46c-4c38-a6e4-a15bea355fa0","Type":"ContainerStarted","Data":"b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"}
Jul 14 00:07:13.638467 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:13.638414    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" podStartSLOduration=1.638395761 podStartE2EDuration="1.638395761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:07:13.636519832 +0000 UTC m=+1685.434420530" watchObservedRunningTime="2026-07-14 00:07:13.638395761 +0000 UTC m=+1685.436296458"
Jul 14 00:07:14.443025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:14.442987    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:14.448740 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:14.448713    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:14.621963 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:14.621930    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:14.623164 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:14.623140    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:16.078143 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:16.078099    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:07:24.671769 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:24.671670    2550 generic.go:356] "Generic (PLEG): container finished" podID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerID="b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe" exitCode=0
Jul 14 00:07:24.671769 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:24.671743    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" event={"ID":"30079938-c46c-4c38-a6e4-a15bea355fa0","Type":"ContainerDied","Data":"b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"}
Jul 14 00:07:24.672324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:24.672201    2550 scope.go:117] "RemoveContainer" containerID="b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"
Jul 14 00:07:24.673304 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:24.673274    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:07:24.673405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:24.673346    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:07:24.673405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:24.673391    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:07:25.678411 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:25.678368    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" event={"ID":"30079938-c46c-4c38-a6e4-a15bea355fa0","Type":"ContainerStarted","Data":"a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c"}
Jul 14 00:07:26.443404 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:26.443356    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:26.444816 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:26.444785    2550 patch_prober.go:28] interesting pod/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 14 00:07:26.444995 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:26.444846    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 14 00:07:27.443948 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:27.443878    2550 patch_prober.go:28] interesting pod/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 14 00:07:27.444370 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:27.443985    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 14 00:07:28.443347 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:28.443304    2550 patch_prober.go:28] interesting pod/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 14 00:07:28.443555 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:28.443367    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 14 00:07:29.443741 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:29.443701    2550 patch_prober.go:28] interesting pod/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 14 00:07:29.444186 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:29.443762    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 14 00:07:29.621976 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:29.621865    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:30.447913 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:30.447857    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:07:30.449100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:07:30.449077    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:08:20.873513 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:20.873462    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:08:32.988570 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:32.988529    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"]
Jul 14 00:08:32.989133 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:32.988753    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy" containerID="cri-o://a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c" gracePeriod=30
Jul 14 00:08:38.250721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.250649    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:08:38.331863 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.331832    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-data\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332072 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.331871    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-podinfo\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332072 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.331945    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30079938-c46c-4c38-a6e4-a15bea355fa0-istiod-ca-cert\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332072 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.331980    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-envoy\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332072 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332052    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-credential-socket\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332292 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332083    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d28r\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-kube-api-access-9d28r\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332292 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332126    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-certs\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332292 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332170    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-token\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332292 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332205    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-socket\") pod \"30079938-c46c-4c38-a6e4-a15bea355fa0\" (UID: \"30079938-c46c-4c38-a6e4-a15bea355fa0\") "
Jul 14 00:08:38.332484 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332366    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30079938-c46c-4c38-a6e4-a15bea355fa0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:08:38.332484 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332394    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:08:38.332591 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332531    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-credential-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.332591 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332550    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30079938-c46c-4c38-a6e4-a15bea355fa0-istiod-ca-cert\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.332591 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332545    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:08:38.332591 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332571    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:08:38.332793 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.332736    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-data" (OuterVolumeSpecName: "istio-data") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:08:38.334727 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.334695    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:08:38.334868 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.334786    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 00:08:38.335188 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.335126    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-token" (OuterVolumeSpecName: "istio-token") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:08:38.335260 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.335227    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-kube-api-access-9d28r" (OuterVolumeSpecName: "kube-api-access-9d28r") pod "30079938-c46c-4c38-a6e4-a15bea355fa0" (UID: "30079938-c46c-4c38-a6e4-a15bea355fa0"). InnerVolumeSpecName "kube-api-access-9d28r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:08:38.433988 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.433947    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-podinfo\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.433988 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.433982    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-envoy\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.433988 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.433992    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d28r\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-kube-api-access-9d28r\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.433988 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.434001    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.434324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.434010    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-token\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.434324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.434018    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-workload-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:38.434324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:38.434025    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30079938-c46c-4c38-a6e4-a15bea355fa0-istio-data\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:08:39.051271 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.051171    2550 generic.go:356] "Generic (PLEG): container finished" podID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerID="a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c" exitCode=0
Jul 14 00:08:39.051271 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.051237    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"
Jul 14 00:08:39.051500 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.051271    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" event={"ID":"30079938-c46c-4c38-a6e4-a15bea355fa0","Type":"ContainerDied","Data":"a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c"}
Jul 14 00:08:39.051500 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.051348    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5" event={"ID":"30079938-c46c-4c38-a6e4-a15bea355fa0","Type":"ContainerDied","Data":"c2073da34e8dce93eb51f2acea1ad730e91855389da3a18f2852137bd4792bc1"}
Jul 14 00:08:39.051500 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.051383    2550 scope.go:117] "RemoveContainer" containerID="a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c"
Jul 14 00:08:39.062251 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.062221    2550 scope.go:117] "RemoveContainer" containerID="b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"
Jul 14 00:08:39.070448 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.070411    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"]
Jul 14 00:08:39.072512 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.072488    2550 scope.go:117] "RemoveContainer" containerID="a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c"
Jul 14 00:08:39.073741 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:08:39.073110    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c\": container with ID starting with a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c not found: ID does not exist" containerID="a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c"
Jul 14 00:08:39.073877 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.073769    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c"} err="failed to get container status \"a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c\": rpc error: code = NotFound desc = could not find container \"a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c\": container with ID starting with a828ed6c59f0d63b8e8663dd6467e73a6178f8165b575b387f14ba12c4220b0c not found: ID does not exist"
Jul 14 00:08:39.073877 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.073802    2550 scope.go:117] "RemoveContainer" containerID="b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"
Jul 14 00:08:39.074182 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:08:39.074155    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe\": container with ID starting with b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe not found: ID does not exist" containerID="b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"
Jul 14 00:08:39.074281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.074194    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe"} err="failed to get container status \"b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe\": rpc error: code = NotFound desc = could not find container \"b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe\": container with ID starting with b0c71b6749d6ba0db379125844482f5acaf3942fd19f3ed9e210730407383ebe not found: ID does not exist"
Jul 14 00:08:39.075880 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:39.075859    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bb015d31-openshift-default-8f475f49f-h2tz5"]
Jul 14 00:08:40.777734 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:08:40.777698    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" path="/var/lib/kubelet/pods/30079938-c46c-4c38-a6e4-a15bea355fa0/volumes"
Jul 14 00:09:08.770696 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:09:08.770661    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:09:08.782470 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:09:08.782437    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:09:08.787077 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:09:08.787052    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:09:08.790636 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:09:08.790610    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:09:55.688629 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:09:55.688587    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:10:04.587057 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:04.587014    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:10:16.291993 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:16.291949    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:10:22.598443 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.598406    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"]
Jul 14 00:10:22.598883 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.598806    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy"
Jul 14 00:10:22.598883 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.598817    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy"
Jul 14 00:10:22.598883 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.598879    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy"
Jul 14 00:10:22.599045 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.598888    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy"
Jul 14 00:10:22.601991 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.601968    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:22.604534 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.604506    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 00:10:22.604625 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.604512    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vzxsj\""
Jul 14 00:10:22.609662 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.609633    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"]
Jul 14 00:10:22.655125 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.655084    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fd1b1a-1ee1-4340-9157-78be3df03587-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45127b63-prmtx\" (UID: \"82fd1b1a-1ee1-4340-9157-78be3df03587\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:22.756329 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.756288    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fd1b1a-1ee1-4340-9157-78be3df03587-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45127b63-prmtx\" (UID: \"82fd1b1a-1ee1-4340-9157-78be3df03587\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:22.757012 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.756995    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fd1b1a-1ee1-4340-9157-78be3df03587-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45127b63-prmtx\" (UID: \"82fd1b1a-1ee1-4340-9157-78be3df03587\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:22.914379 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:22.914277    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:23.060511 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:23.060380    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"]
Jul 14 00:10:23.063552 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:10:23.063511    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82fd1b1a_1ee1_4340_9157_78be3df03587.slice/crio-0f80d094c7563ab9930fe3fb3c8f38621132af8a491f084a091abbdd135733ec WatchSource:0}: Error finding container 0f80d094c7563ab9930fe3fb3c8f38621132af8a491f084a091abbdd135733ec: Status 404 returned error can't find the container with id 0f80d094c7563ab9930fe3fb3c8f38621132af8a491f084a091abbdd135733ec
Jul 14 00:10:23.556549 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:23.556444    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx" event={"ID":"82fd1b1a-1ee1-4340-9157-78be3df03587","Type":"ContainerStarted","Data":"f02556c1d9c6969c88e80dad9c21636a53910e287e8555642ebb8e8755ee5e6c"}
Jul 14 00:10:23.556549 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:23.556502    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx" event={"ID":"82fd1b1a-1ee1-4340-9157-78be3df03587","Type":"ContainerStarted","Data":"0f80d094c7563ab9930fe3fb3c8f38621132af8a491f084a091abbdd135733ec"}
Jul 14 00:10:23.571921 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:23.571837    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx" podStartSLOduration=1.5718191529999999 podStartE2EDuration="1.571819153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:10:23.57069798 +0000 UTC m=+1875.368598704" watchObservedRunningTime="2026-07-14 00:10:23.571819153 +0000 UTC m=+1875.369719851"
Jul 14 00:10:24.566797 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:24.566759    2550 generic.go:356] "Generic (PLEG): container finished" podID="82fd1b1a-1ee1-4340-9157-78be3df03587" containerID="f02556c1d9c6969c88e80dad9c21636a53910e287e8555642ebb8e8755ee5e6c" exitCode=0
Jul 14 00:10:24.567220 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:24.566842    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx" event={"ID":"82fd1b1a-1ee1-4340-9157-78be3df03587","Type":"ContainerDied","Data":"f02556c1d9c6969c88e80dad9c21636a53910e287e8555642ebb8e8755ee5e6c"}
Jul 14 00:10:25.708428 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:25.708393    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:25.778588 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:25.778549    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fd1b1a-1ee1-4340-9157-78be3df03587-maas-api-service-ca\") pod \"82fd1b1a-1ee1-4340-9157-78be3df03587\" (UID: \"82fd1b1a-1ee1-4340-9157-78be3df03587\") "
Jul 14 00:10:25.778986 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:25.778961    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82fd1b1a-1ee1-4340-9157-78be3df03587-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82fd1b1a-1ee1-4340-9157-78be3df03587" (UID: "82fd1b1a-1ee1-4340-9157-78be3df03587"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:10:25.879584 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:25.879492    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82fd1b1a-1ee1-4340-9157-78be3df03587-maas-api-service-ca\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:10:26.577622 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:26.577588    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx"
Jul 14 00:10:26.577622 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:26.577616    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45127b63-prmtx" event={"ID":"82fd1b1a-1ee1-4340-9157-78be3df03587","Type":"ContainerDied","Data":"0f80d094c7563ab9930fe3fb3c8f38621132af8a491f084a091abbdd135733ec"}
Jul 14 00:10:26.578027 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:26.577665    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f80d094c7563ab9930fe3fb3c8f38621132af8a491f084a091abbdd135733ec"
Jul 14 00:10:35.882008 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:10:35.881919    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:11:17.284422 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:17.284375    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:11:20.545353 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.545320    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"]
Jul 14 00:11:20.545735 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.545703    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy"
Jul 14 00:11:20.545735 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.545715    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="30079938-c46c-4c38-a6e4-a15bea355fa0" containerName="istio-proxy"
Jul 14 00:11:20.545735 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.545724    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82fd1b1a-1ee1-4340-9157-78be3df03587" containerName="revoke-keys"
Jul 14 00:11:20.545735 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.545730    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="82fd1b1a-1ee1-4340-9157-78be3df03587" containerName="revoke-keys"
Jul 14 00:11:20.545856 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.545796    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="82fd1b1a-1ee1-4340-9157-78be3df03587" containerName="revoke-keys"
Jul 14 00:11:20.548686 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.548669    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.552135 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.552102    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ca846028\"/\"default-dockercfg-xjjg6\""
Jul 14 00:11:20.552294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.552175    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ca846028\"/\"openshift-service-ca.crt\""
Jul 14 00:11:20.552294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.552103    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ca846028\"/\"e2e-lifecycle-model-ca846028-kserve-self-signed-certs\""
Jul 14 00:11:20.552294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.552119    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ca846028\"/\"kube-root-ca.crt\""
Jul 14 00:11:20.559102 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.559077    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"]
Jul 14 00:11:20.673412 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.673366    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-tmp-dir\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.673412 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.673420    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-home\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.673650 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.673507    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk24f\" (UniqueName: \"kubernetes.io/projected/4b72269f-69f1-4205-82cd-bf4a60450796-kube-api-access-jk24f\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.673650 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.673544    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-dshm\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.673650 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.673581    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b72269f-69f1-4205-82cd-bf4a60450796-tls-certs\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.673650 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.673604    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-model-cache\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774083 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774044    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-tmp-dir\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774097    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-home\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774157    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk24f\" (UniqueName: \"kubernetes.io/projected/4b72269f-69f1-4205-82cd-bf4a60450796-kube-api-access-jk24f\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774183    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-dshm\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774224    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b72269f-69f1-4205-82cd-bf4a60450796-tls-certs\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774253    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-model-cache\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774540 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774518    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-tmp-dir\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774605 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774563    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-home\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.774713 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.774684    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-model-cache\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.776990 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.776968    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-dshm\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.777084 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.777048    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b72269f-69f1-4205-82cd-bf4a60450796-tls-certs\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.782415 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.782392    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk24f\" (UniqueName: \"kubernetes.io/projected/4b72269f-69f1-4205-82cd-bf4a60450796-kube-api-access-jk24f\") pod \"e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") " pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:20.858867 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:20.858780    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:21.003341 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:21.003311    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"]
Jul 14 00:11:21.006123 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:11:21.006086    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b72269f_69f1_4205_82cd_bf4a60450796.slice/crio-107610ed827fb8040839a71a41bbb543d96b58535c6f11efdac1674f153890f3 WatchSource:0}: Error finding container 107610ed827fb8040839a71a41bbb543d96b58535c6f11efdac1674f153890f3: Status 404 returned error can't find the container with id 107610ed827fb8040839a71a41bbb543d96b58535c6f11efdac1674f153890f3
Jul 14 00:11:21.008603 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:21.008585    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 00:11:21.839176 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:21.839133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x" event={"ID":"4b72269f-69f1-4205-82cd-bf4a60450796","Type":"ContainerStarted","Data":"107610ed827fb8040839a71a41bbb543d96b58535c6f11efdac1674f153890f3"}
Jul 14 00:11:22.845764 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:22.845728    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x" event={"ID":"4b72269f-69f1-4205-82cd-bf4a60450796","Type":"ContainerStarted","Data":"6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8"}
Jul 14 00:11:22.863954 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:22.863882    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x" podStartSLOduration=1.806015395 podStartE2EDuration="2.863863584s" totalImagesPullingTime="1.057848189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:11:22.861791991 +0000 UTC m=+1934.659692689" watchObservedRunningTime="2026-07-14 00:11:22.863863584 +0000 UTC m=+1934.661764282"
Jul 14 00:11:30.859342 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:30.859300    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:30.859342 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:30.859353    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:30.872182 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:30.872155    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:30.896541 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:30.896511    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:11:32.893922 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:32.893865    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:11:39.481557 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:39.481506    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:11:57.880122 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:11:57.880078    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:12:03.806588 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:03.806508    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"]
Jul 14 00:12:08.844894 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:08.844861    2550 scope.go:117] "RemoveContainer" containerID="bf9d6117d3aae98e0ad653b3bea6dcf94773713850104bf9e1ce1423f0aa21df"
Jul 14 00:12:18.828085 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:18.828011    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x" podUID="4b72269f-69f1-4205-82cd-bf4a60450796" containerName="main" containerID="cri-o://6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8" gracePeriod=45
Jul 14 00:12:19.091418 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.091356    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:12:19.116765 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.116732    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b72269f-69f1-4205-82cd-bf4a60450796-tls-certs\") pod \"4b72269f-69f1-4205-82cd-bf4a60450796\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") "
Jul 14 00:12:19.116974 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.116775    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-dshm\") pod \"4b72269f-69f1-4205-82cd-bf4a60450796\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") "
Jul 14 00:12:19.116974 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.116798    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-model-cache\") pod \"4b72269f-69f1-4205-82cd-bf4a60450796\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") "
Jul 14 00:12:19.116974 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.116834    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk24f\" (UniqueName: \"kubernetes.io/projected/4b72269f-69f1-4205-82cd-bf4a60450796-kube-api-access-jk24f\") pod \"4b72269f-69f1-4205-82cd-bf4a60450796\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") "
Jul 14 00:12:19.116974 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.116861    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-tmp-dir\") pod \"4b72269f-69f1-4205-82cd-bf4a60450796\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") "
Jul 14 00:12:19.117214 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.116987    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-home\") pod \"4b72269f-69f1-4205-82cd-bf4a60450796\" (UID: \"4b72269f-69f1-4205-82cd-bf4a60450796\") "
Jul 14 00:12:19.117214 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.117145    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b72269f-69f1-4205-82cd-bf4a60450796" (UID: "4b72269f-69f1-4205-82cd-bf4a60450796"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:12:19.117318 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.117294    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:12:19.117391 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.117338    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-home" (OuterVolumeSpecName: "home") pod "4b72269f-69f1-4205-82cd-bf4a60450796" (UID: "4b72269f-69f1-4205-82cd-bf4a60450796"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:12:19.117391 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.117336    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b72269f-69f1-4205-82cd-bf4a60450796" (UID: "4b72269f-69f1-4205-82cd-bf4a60450796"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:12:19.119640 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.119608    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b72269f-69f1-4205-82cd-bf4a60450796-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b72269f-69f1-4205-82cd-bf4a60450796" (UID: "4b72269f-69f1-4205-82cd-bf4a60450796"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:12:19.119781 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.119661    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-dshm" (OuterVolumeSpecName: "dshm") pod "4b72269f-69f1-4205-82cd-bf4a60450796" (UID: "4b72269f-69f1-4205-82cd-bf4a60450796"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:12:19.120113 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.120086    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b72269f-69f1-4205-82cd-bf4a60450796-kube-api-access-jk24f" (OuterVolumeSpecName: "kube-api-access-jk24f") pod "4b72269f-69f1-4205-82cd-bf4a60450796" (UID: "4b72269f-69f1-4205-82cd-bf4a60450796"). InnerVolumeSpecName "kube-api-access-jk24f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:12:19.125473 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.125444    2550 generic.go:356] "Generic (PLEG): container finished" podID="4b72269f-69f1-4205-82cd-bf4a60450796" containerID="6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8" exitCode=0
Jul 14 00:12:19.125618 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.125493    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"
Jul 14 00:12:19.125618 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.125502    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x" event={"ID":"4b72269f-69f1-4205-82cd-bf4a60450796","Type":"ContainerDied","Data":"6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8"}
Jul 14 00:12:19.125618 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.125540    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x" event={"ID":"4b72269f-69f1-4205-82cd-bf4a60450796","Type":"ContainerDied","Data":"107610ed827fb8040839a71a41bbb543d96b58535c6f11efdac1674f153890f3"}
Jul 14 00:12:19.125618 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.125559    2550 scope.go:117] "RemoveContainer" containerID="6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8"
Jul 14 00:12:19.143013 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.142987    2550 scope.go:117] "RemoveContainer" containerID="6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8"
Jul 14 00:12:19.143408 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:12:19.143382    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8\": container with ID starting with 6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8 not found: ID does not exist" containerID="6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8"
Jul 14 00:12:19.143506 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.143416    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8"} err="failed to get container status \"6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8\": rpc error: code = NotFound desc = could not find container \"6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8\": container with ID starting with 6253098df176ffca845f76014f6e40192a30f103cf03ac379e25934fc1f1f1d8 not found: ID does not exist"
Jul 14 00:12:19.156252 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.156220    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"]
Jul 14 00:12:19.158585 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.158559    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ca846028/e2e-lifecycle-model-ca846028-kserve-5795fb4c9f-8nf9x"]
Jul 14 00:12:19.218620 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.218586    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:12:19.218620 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.218610    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b72269f-69f1-4205-82cd-bf4a60450796-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:12:19.218620 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.218619    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:12:19.218620 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.218627    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk24f\" (UniqueName: \"kubernetes.io/projected/4b72269f-69f1-4205-82cd-bf4a60450796-kube-api-access-jk24f\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:12:19.218880 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:19.218661    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b72269f-69f1-4205-82cd-bf4a60450796-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:12:20.776868 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:20.776829    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b72269f-69f1-4205-82cd-bf4a60450796" path="/var/lib/kubelet/pods/4b72269f-69f1-4205-82cd-bf4a60450796/volumes"
Jul 14 00:12:55.189246 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:12:55.189188    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:13:07.484173 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:07.484136    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:13:53.039341 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.039237    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"]
Jul 14 00:13:53.039943 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.039771    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b72269f-69f1-4205-82cd-bf4a60450796" containerName="main"
Jul 14 00:13:53.039943 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.039793    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b72269f-69f1-4205-82cd-bf4a60450796" containerName="main"
Jul 14 00:13:53.039943 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.039934    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b72269f-69f1-4205-82cd-bf4a60450796" containerName="main"
Jul 14 00:13:53.043210 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.043186    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.045440 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.045399    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-00e043-openshift-default-dockercfg-kj5ts\""
Jul 14 00:13:53.053085 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.053063    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"]
Jul 14 00:13:53.073198 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073159    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-socket\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073198 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073205    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-podinfo\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073463 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073224    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-token\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073463 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073308    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-envoy\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073463 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073351    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-credential-socket\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073463 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073406    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-certs\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073463 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073437    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqcj\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-kube-api-access-qhqcj\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073638 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073479    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-data\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.073638 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.073510    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5acd4f5-bad5-4ebf-b891-0445867abe97-istiod-ca-cert\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174125 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174092    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-data\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174125 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174139    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5acd4f5-bad5-4ebf-b891-0445867abe97-istiod-ca-cert\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174164    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-socket\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174191    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-podinfo\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174211    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-token\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174244    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-envoy\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.174414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174270    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-credential-socket\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174575    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-data\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174703    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-socket\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174717    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-certs\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174760    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqcj\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-kube-api-access-qhqcj\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174798    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-credential-socket\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.174988    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5acd4f5-bad5-4ebf-b891-0445867abe97-istiod-ca-cert\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.175321 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.175102    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-certs\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.176928 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.176873    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-envoy\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.177160 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.177094    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-podinfo\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.182737 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.182704    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqcj\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-kube-api-access-qhqcj\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.182830 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.182743    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-token\") pod \"e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") " pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.355263 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.355160    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:53.501960 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.501893    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"]
Jul 14 00:13:53.504601 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:13:53.504566    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5acd4f5_bad5_4ebf_b891_0445867abe97.slice/crio-a419947b215282672ecf82a1afaffa7132cd021a201c2ce8f4c9ce7ea8688b20 WatchSource:0}: Error finding container a419947b215282672ecf82a1afaffa7132cd021a201c2ce8f4c9ce7ea8688b20: Status 404 returned error can't find the container with id a419947b215282672ecf82a1afaffa7132cd021a201c2ce8f4c9ce7ea8688b20
Jul 14 00:13:53.506724 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.506690    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:13:53.506811 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.506777    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:13:53.506811 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.506808    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:13:53.578991 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.578954    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" event={"ID":"a5acd4f5-bad5-4ebf-b891-0445867abe97","Type":"ContainerStarted","Data":"aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"}
Jul 14 00:13:53.579131 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.579024    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" event={"ID":"a5acd4f5-bad5-4ebf-b891-0445867abe97","Type":"ContainerStarted","Data":"a419947b215282672ecf82a1afaffa7132cd021a201c2ce8f4c9ce7ea8688b20"}
Jul 14 00:13:53.597625 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:53.597552    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podStartSLOduration=0.597529115 podStartE2EDuration="597.529115ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:13:53.597149755 +0000 UTC m=+2085.395050473" watchObservedRunningTime="2026-07-14 00:13:53.597529115 +0000 UTC m=+2085.395429816"
Jul 14 00:13:54.355826 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:54.355787    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:54.361437 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:54.361399    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:54.584601 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:54.584563    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:54.585832 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:54.585804    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:13:55.698149 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:13:55.698117    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:14:05.741916 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.741862    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"]
Jul 14 00:14:05.745665 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.745644    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:05.748948 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.748919    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-36055a\"/\"default-dockercfg-pg44s\""
Jul 14 00:14:05.749096 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.748927    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-36055a\"/\"kube-root-ca.crt\""
Jul 14 00:14:05.749096 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.748929    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-36055a\"/\"openshift-service-ca.crt\""
Jul 14 00:14:05.749096 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.749033    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-36055a\"/\"test-model-36055a-kserve-self-signed-certs\""
Jul 14 00:14:05.757003 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.756982    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"]
Jul 14 00:14:05.898795 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.898739    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-model-cache\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:05.899236 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.899199    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-tmp-dir\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:05.899416 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.899395    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc8wg\" (UniqueName: \"kubernetes.io/projected/1911f356-37a5-4f22-a24c-d0313b024e67-kube-api-access-gc8wg\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:05.899597 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.899566    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-home\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:05.899752 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.899737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-dshm\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:05.900018 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:05.900000    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1911f356-37a5-4f22-a24c-d0313b024e67-tls-certs\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.001198 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.001081    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-home\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.001509 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.001484    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-dshm\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.001722 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.001706    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1911f356-37a5-4f22-a24c-d0313b024e67-tls-certs\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.001878 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.001862    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-model-cache\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.002034 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.002018    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-tmp-dir\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.002179 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.001732    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-home\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.002297 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.002196    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-model-cache\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.002414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.002220    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc8wg\" (UniqueName: \"kubernetes.io/projected/1911f356-37a5-4f22-a24c-d0313b024e67-kube-api-access-gc8wg\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.004994 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.002726    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-tmp-dir\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.007236 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.006456    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1911f356-37a5-4f22-a24c-d0313b024e67-tls-certs\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.007236 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.007153    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-dshm\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.011703 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.011680    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc8wg\" (UniqueName: \"kubernetes.io/projected/1911f356-37a5-4f22-a24c-d0313b024e67-kube-api-access-gc8wg\") pod \"test-model-36055a-kserve-59d6d569b9-6pjnc\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") " pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.056087 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.056046    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:06.241126 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.241079    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"]
Jul 14 00:14:06.243832 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:14:06.243792    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1911f356_37a5_4f22_a24c_d0313b024e67.slice/crio-7f964c88f2b58215933e388d6fee165abd508d19090f3cda310fe6f756f4d45d WatchSource:0}: Error finding container 7f964c88f2b58215933e388d6fee165abd508d19090f3cda310fe6f756f4d45d: Status 404 returned error can't find the container with id 7f964c88f2b58215933e388d6fee165abd508d19090f3cda310fe6f756f4d45d
Jul 14 00:14:06.641513 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.641469    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc" event={"ID":"1911f356-37a5-4f22-a24c-d0313b024e67","Type":"ContainerStarted","Data":"6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4"}
Jul 14 00:14:06.641716 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.641524    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc" event={"ID":"1911f356-37a5-4f22-a24c-d0313b024e67","Type":"ContainerStarted","Data":"7f964c88f2b58215933e388d6fee165abd508d19090f3cda310fe6f756f4d45d"}
Jul 14 00:14:06.659520 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:06.659457    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc" podStartSLOduration=1.6594392999999998 podStartE2EDuration="1.6594393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:14:06.65855554 +0000 UTC m=+2098.456456238" watchObservedRunningTime="2026-07-14 00:14:06.6594393 +0000 UTC m=+2098.457340000"
Jul 14 00:14:08.817023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:08.816992    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:14:08.823619 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:08.823589    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:14:08.825373 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:08.825351    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:14:08.831879 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:08.831856    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:14:09.779465 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:09.779427    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:14:10.626138 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:10.626098    2550 patch_prober.go:28] interesting pod/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 14 00:14:10.626554 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:10.626161    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 14 00:14:12.670627 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:12.670591    2550 generic.go:356] "Generic (PLEG): container finished" podID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerID="aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31" exitCode=0
Jul 14 00:14:12.671101 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:12.670666    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" event={"ID":"a5acd4f5-bad5-4ebf-b891-0445867abe97","Type":"ContainerDied","Data":"aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"}
Jul 14 00:14:12.671152 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:12.671118    2550 scope.go:117] "RemoveContainer" containerID="aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"
Jul 14 00:14:12.672222 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:12.672186    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:14:12.672291 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:12.672265    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:14:12.672327 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:12.672297    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:14:13.678091 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:13.678056    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" event={"ID":"a5acd4f5-bad5-4ebf-b891-0445867abe97","Type":"ContainerStarted","Data":"7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980"}
Jul 14 00:14:14.355647 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:14.355598    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:14:14.356032 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:14.356008    2550 patch_prober.go:28] interesting pod/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 14 00:14:14.356108 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:14.356062    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 14 00:14:15.356100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:15.356055    2550 patch_prober.go:28] interesting pod/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 14 00:14:15.356514 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:15.356122    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 14 00:14:16.056501 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:16.056457    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:16.056760 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:16.056527    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:16.069979 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:16.069936    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:16.356421 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:16.356322    2550 patch_prober.go:28] interesting pod/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 14 00:14:16.356421 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:16.356385    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 14 00:14:16.716087 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:16.716000    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:17.355655 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:17.355617    2550 patch_prober.go:28] interesting pod/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 14 00:14:17.355894 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:17.355680    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 14 00:14:18.360845 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:18.360807    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:14:18.361344 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:18.361107    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:14:18.361987 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:18.361971    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:14:23.344096 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.344061    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"]
Jul 14 00:14:23.347315 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.347296    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.351027 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.351002    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-00e043\"/\"kube-root-ca.crt\""
Jul 14 00:14:23.351191 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.351064    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-00e043\"/\"openshift-service-ca.crt\""
Jul 14 00:14:23.351191 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.351088    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-00e043\"/\"test-model-00e043-kserve-self-signed-certs\""
Jul 14 00:14:23.351191 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.351103    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-00e043\"/\"default-dockercfg-72glf\""
Jul 14 00:14:23.358326 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.358299    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"]
Jul 14 00:14:23.385543 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.385509    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:14:23.391355 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.391325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-home\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.391488 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.391377    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tls-certs\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.391488 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.391404    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ztmk\" (UniqueName: \"kubernetes.io/projected/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-kube-api-access-4ztmk\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.391488 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.391436    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-dshm\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.391488 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.391476    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tmp-dir\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.391636 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.391512    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-model-cache\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493014 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.492978    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-home\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493213 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493034    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tls-certs\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493213 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493058    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ztmk\" (UniqueName: \"kubernetes.io/projected/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-kube-api-access-4ztmk\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493213 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493080    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-dshm\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493213 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493116    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tmp-dir\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493455 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493311    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-model-cache\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493524 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493481    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-home\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493584 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493521    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tmp-dir\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.493794 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.493770    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-model-cache\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.495704 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.495685    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-dshm\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.496005 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.495989    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tls-certs\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.501674 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.501652    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ztmk\" (UniqueName: \"kubernetes.io/projected/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-kube-api-access-4ztmk\") pod \"test-model-00e043-kserve-6fd5449fb9-jkkjp\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") " pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.657267 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.657165    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:23.798686 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:23.798662    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"]
Jul 14 00:14:23.800720 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:14:23.800685    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c48fcc5_6107_44a7_aeb5_56f5ed191a97.slice/crio-c0ac6ec3048fed4213ed58ec75637f1695447f9bc247c8f01e9a33f2f8cdbe36 WatchSource:0}: Error finding container c0ac6ec3048fed4213ed58ec75637f1695447f9bc247c8f01e9a33f2f8cdbe36: Status 404 returned error can't find the container with id c0ac6ec3048fed4213ed58ec75637f1695447f9bc247c8f01e9a33f2f8cdbe36
Jul 14 00:14:24.746001 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:24.745960    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp" event={"ID":"6c48fcc5-6107-44a7-aeb5-56f5ed191a97","Type":"ContainerStarted","Data":"f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe"}
Jul 14 00:14:24.746518 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:24.746014    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp" event={"ID":"6c48fcc5-6107-44a7-aeb5-56f5ed191a97","Type":"ContainerStarted","Data":"c0ac6ec3048fed4213ed58ec75637f1695447f9bc247c8f01e9a33f2f8cdbe36"}
Jul 14 00:14:24.762681 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:24.762624    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp" podStartSLOduration=1.7626079890000002 podStartE2EDuration="1.762607989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:14:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:14:24.761129879 +0000 UTC m=+2116.559030577" watchObservedRunningTime="2026-07-14 00:14:24.762607989 +0000 UTC m=+2116.560508680"
Jul 14 00:14:33.657437 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:33.657381    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:33.657843 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:33.657663    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:33.673490 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:33.673453    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:33.804707 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:33.804676    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:41.874467 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:41.874426    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:14:43.288441 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:43.288405    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"]
Jul 14 00:14:44.228628 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:44.228586    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"]
Jul 14 00:14:53.680023 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:53.679988    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:14:58.310478 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.310406    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc" podUID="1911f356-37a5-4f22-a24c-d0313b024e67" containerName="main" containerID="cri-o://6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4" gracePeriod=45
Jul 14 00:14:58.574676 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.574598    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:58.727597 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.727560    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-dshm\") pod \"1911f356-37a5-4f22-a24c-d0313b024e67\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") "
Jul 14 00:14:58.727828 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.727619    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc8wg\" (UniqueName: \"kubernetes.io/projected/1911f356-37a5-4f22-a24c-d0313b024e67-kube-api-access-gc8wg\") pod \"1911f356-37a5-4f22-a24c-d0313b024e67\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") "
Jul 14 00:14:58.727828 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.727651    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1911f356-37a5-4f22-a24c-d0313b024e67-tls-certs\") pod \"1911f356-37a5-4f22-a24c-d0313b024e67\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") "
Jul 14 00:14:58.727828 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.727684    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-home\") pod \"1911f356-37a5-4f22-a24c-d0313b024e67\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") "
Jul 14 00:14:58.727828 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.727710    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-tmp-dir\") pod \"1911f356-37a5-4f22-a24c-d0313b024e67\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") "
Jul 14 00:14:58.727828 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.727761    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-model-cache\") pod \"1911f356-37a5-4f22-a24c-d0313b024e67\" (UID: \"1911f356-37a5-4f22-a24c-d0313b024e67\") "
Jul 14 00:14:58.728152 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.728010    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1911f356-37a5-4f22-a24c-d0313b024e67" (UID: "1911f356-37a5-4f22-a24c-d0313b024e67"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:58.728152 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.728026    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-home" (OuterVolumeSpecName: "home") pod "1911f356-37a5-4f22-a24c-d0313b024e67" (UID: "1911f356-37a5-4f22-a24c-d0313b024e67"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:58.728152 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.728127    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-model-cache" (OuterVolumeSpecName: "model-cache") pod "1911f356-37a5-4f22-a24c-d0313b024e67" (UID: "1911f356-37a5-4f22-a24c-d0313b024e67"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:58.728288 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.728164    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:58.728288 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.728182    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:58.730338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.730305    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1911f356-37a5-4f22-a24c-d0313b024e67-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1911f356-37a5-4f22-a24c-d0313b024e67" (UID: "1911f356-37a5-4f22-a24c-d0313b024e67"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:14:58.730457 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.730350    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1911f356-37a5-4f22-a24c-d0313b024e67-kube-api-access-gc8wg" (OuterVolumeSpecName: "kube-api-access-gc8wg") pod "1911f356-37a5-4f22-a24c-d0313b024e67" (UID: "1911f356-37a5-4f22-a24c-d0313b024e67"). InnerVolumeSpecName "kube-api-access-gc8wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:14:58.730457 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.730381    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-dshm" (OuterVolumeSpecName: "dshm") pod "1911f356-37a5-4f22-a24c-d0313b024e67" (UID: "1911f356-37a5-4f22-a24c-d0313b024e67"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:58.829432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.829323    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:58.829432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.829356    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc8wg\" (UniqueName: \"kubernetes.io/projected/1911f356-37a5-4f22-a24c-d0313b024e67-kube-api-access-gc8wg\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:58.829432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.829370    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1911f356-37a5-4f22-a24c-d0313b024e67-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:58.829432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.829378    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1911f356-37a5-4f22-a24c-d0313b024e67-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:58.920134 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.920095    2550 generic.go:356] "Generic (PLEG): container finished" podID="1911f356-37a5-4f22-a24c-d0313b024e67" containerID="6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4" exitCode=0
Jul 14 00:14:58.920329 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.920148    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"
Jul 14 00:14:58.920329 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.920189    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc" event={"ID":"1911f356-37a5-4f22-a24c-d0313b024e67","Type":"ContainerDied","Data":"6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4"}
Jul 14 00:14:58.920329 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.920241    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc" event={"ID":"1911f356-37a5-4f22-a24c-d0313b024e67","Type":"ContainerDied","Data":"7f964c88f2b58215933e388d6fee165abd508d19090f3cda310fe6f756f4d45d"}
Jul 14 00:14:58.920329 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.920265    2550 scope.go:117] "RemoveContainer" containerID="6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4"
Jul 14 00:14:58.939599 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.939548    2550 scope.go:117] "RemoveContainer" containerID="6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4"
Jul 14 00:14:58.940044 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:14:58.940017    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4\": container with ID starting with 6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4 not found: ID does not exist" containerID="6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4"
Jul 14 00:14:58.940168 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.940050    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4"} err="failed to get container status \"6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4\": rpc error: code = NotFound desc = could not find container \"6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4\": container with ID starting with 6858c1bc87732436b3cab419dffb92f1d1040cb6b7f916108df603f423a7bbc4 not found: ID does not exist"
Jul 14 00:14:58.940168 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.940140    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"]
Jul 14 00:14:58.941913 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:58.941870    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-36055a/test-model-36055a-kserve-59d6d569b9-6pjnc"]
Jul 14 00:14:59.250397 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.250289    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp" podUID="6c48fcc5-6107-44a7-aeb5-56f5ed191a97" containerName="main" containerID="cri-o://f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe" gracePeriod=45
Jul 14 00:14:59.525858 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.525829    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:59.534987 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.534963    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-model-cache\") pod \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") "
Jul 14 00:14:59.535100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535002    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tmp-dir\") pod \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") "
Jul 14 00:14:59.535100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535028    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ztmk\" (UniqueName: \"kubernetes.io/projected/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-kube-api-access-4ztmk\") pod \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") "
Jul 14 00:14:59.535100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535057    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-home\") pod \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") "
Jul 14 00:14:59.535100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535084    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tls-certs\") pod \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") "
Jul 14 00:14:59.535100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535099    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-dshm\") pod \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\" (UID: \"6c48fcc5-6107-44a7-aeb5-56f5ed191a97\") "
Jul 14 00:14:59.535350 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535268    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-model-cache" (OuterVolumeSpecName: "model-cache") pod "6c48fcc5-6107-44a7-aeb5-56f5ed191a97" (UID: "6c48fcc5-6107-44a7-aeb5-56f5ed191a97"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:59.535350 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535298    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6c48fcc5-6107-44a7-aeb5-56f5ed191a97" (UID: "6c48fcc5-6107-44a7-aeb5-56f5ed191a97"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:59.535438 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535359    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-home" (OuterVolumeSpecName: "home") pod "6c48fcc5-6107-44a7-aeb5-56f5ed191a97" (UID: "6c48fcc5-6107-44a7-aeb5-56f5ed191a97"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:59.535487 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535434    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:59.535487 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.535479    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:59.537625 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.537589    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-dshm" (OuterVolumeSpecName: "dshm") pod "6c48fcc5-6107-44a7-aeb5-56f5ed191a97" (UID: "6c48fcc5-6107-44a7-aeb5-56f5ed191a97"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:14:59.537721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.537652    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-kube-api-access-4ztmk" (OuterVolumeSpecName: "kube-api-access-4ztmk") pod "6c48fcc5-6107-44a7-aeb5-56f5ed191a97" (UID: "6c48fcc5-6107-44a7-aeb5-56f5ed191a97"). InnerVolumeSpecName "kube-api-access-4ztmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:14:59.537721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.537651    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6c48fcc5-6107-44a7-aeb5-56f5ed191a97" (UID: "6c48fcc5-6107-44a7-aeb5-56f5ed191a97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:14:59.636893 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.636810    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ztmk\" (UniqueName: \"kubernetes.io/projected/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-kube-api-access-4ztmk\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:59.636893 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.636842    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:59.636893 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.636852    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:59.636893 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.636862    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c48fcc5-6107-44a7-aeb5-56f5ed191a97-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:14:59.927469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.927428    2550 generic.go:356] "Generic (PLEG): container finished" podID="6c48fcc5-6107-44a7-aeb5-56f5ed191a97" containerID="f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe" exitCode=0
Jul 14 00:14:59.927763 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.927474    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"
Jul 14 00:14:59.927763 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.927520    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp" event={"ID":"6c48fcc5-6107-44a7-aeb5-56f5ed191a97","Type":"ContainerDied","Data":"f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe"}
Jul 14 00:14:59.927763 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.927565    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp" event={"ID":"6c48fcc5-6107-44a7-aeb5-56f5ed191a97","Type":"ContainerDied","Data":"c0ac6ec3048fed4213ed58ec75637f1695447f9bc247c8f01e9a33f2f8cdbe36"}
Jul 14 00:14:59.927763 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.927585    2550 scope.go:117] "RemoveContainer" containerID="f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe"
Jul 14 00:14:59.937017 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.936994    2550 scope.go:117] "RemoveContainer" containerID="f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe"
Jul 14 00:14:59.937289 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:14:59.937270    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe\": container with ID starting with f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe not found: ID does not exist" containerID="f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe"
Jul 14 00:14:59.937339 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.937298    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe"} err="failed to get container status \"f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe\": rpc error: code = NotFound desc = could not find container \"f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe\": container with ID starting with f4b362497865ae472a7d8e896616bf0a168874b756d04ce89b92c8bc02ea9dfe not found: ID does not exist"
Jul 14 00:14:59.948726 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.948695    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"]
Jul 14 00:14:59.954288 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:14:59.954262    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-00e043/test-model-00e043-kserve-6fd5449fb9-jkkjp"]
Jul 14 00:15:00.777380 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:00.777342    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1911f356-37a5-4f22-a24c-d0313b024e67" path="/var/lib/kubelet/pods/1911f356-37a5-4f22-a24c-d0313b024e67/volumes"
Jul 14 00:15:00.777788 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:00.777754    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c48fcc5-6107-44a7-aeb5-56f5ed191a97" path="/var/lib/kubelet/pods/6c48fcc5-6107-44a7-aeb5-56f5ed191a97/volumes"
Jul 14 00:15:03.385394 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:03.385359    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:15:24.276176 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:24.276132    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:15:36.238816 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:36.238773    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"]
Jul 14 00:15:36.239456 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:36.239109    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy" containerID="cri-o://7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980" gracePeriod=30
Jul 14 00:15:41.513846 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.513774    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:15:41.529101 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529067    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-token\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529101 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529105    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5acd4f5-bad5-4ebf-b891-0445867abe97-istiod-ca-cert\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529266 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529140    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhqcj\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-kube-api-access-qhqcj\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529266 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529225    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-envoy\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529266 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529252    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-certs\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529285    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-data\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529313    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-podinfo\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529338    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-credential-socket\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529405 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529381    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-socket\") pod \"a5acd4f5-bad5-4ebf-b891-0445867abe97\" (UID: \"a5acd4f5-bad5-4ebf-b891-0445867abe97\") "
Jul 14 00:15:41.529613 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529555    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5acd4f5-bad5-4ebf-b891-0445867abe97-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:15:41.529714 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529650    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:15:41.529786 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529734    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5acd4f5-bad5-4ebf-b891-0445867abe97-istiod-ca-cert\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.529830 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529758    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:15:41.529830 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.529808    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:15:41.530714 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.530675    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:15:41.531662 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.531638    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:15:41.531773 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.531743    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-kube-api-access-qhqcj" (OuterVolumeSpecName: "kube-api-access-qhqcj") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "kube-api-access-qhqcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:15:41.532144 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.532124    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 00:15:41.532317 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.532296    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5acd4f5-bad5-4ebf-b891-0445867abe97" (UID: "a5acd4f5-bad5-4ebf-b891-0445867abe97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:15:41.630692 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630644    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.630692 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630683    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-token\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.630692 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630693    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhqcj\" (UniqueName: \"kubernetes.io/projected/a5acd4f5-bad5-4ebf-b891-0445867abe97-kube-api-access-qhqcj\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.630692 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630706    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-envoy\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.631048 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630715    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-workload-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.631048 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630723    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-data\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.631048 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630730    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5acd4f5-bad5-4ebf-b891-0445867abe97-istio-podinfo\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:41.631048 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:41.630737    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5acd4f5-bad5-4ebf-b891-0445867abe97-credential-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:15:42.134611 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.134570    2550 generic.go:356] "Generic (PLEG): container finished" podID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerID="7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980" exitCode=0
Jul 14 00:15:42.134819 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.134625    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"
Jul 14 00:15:42.134819 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.134664    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" event={"ID":"a5acd4f5-bad5-4ebf-b891-0445867abe97","Type":"ContainerDied","Data":"7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980"}
Jul 14 00:15:42.134819 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.134727    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6" event={"ID":"a5acd4f5-bad5-4ebf-b891-0445867abe97","Type":"ContainerDied","Data":"a419947b215282672ecf82a1afaffa7132cd021a201c2ce8f4c9ce7ea8688b20"}
Jul 14 00:15:42.134819 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.134758    2550 scope.go:117] "RemoveContainer" containerID="7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980"
Jul 14 00:15:42.144757 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.144736    2550 scope.go:117] "RemoveContainer" containerID="aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"
Jul 14 00:15:42.153761 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.153720    2550 scope.go:117] "RemoveContainer" containerID="7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980"
Jul 14 00:15:42.154142 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:15:42.154114    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980\": container with ID starting with 7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980 not found: ID does not exist" containerID="7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980"
Jul 14 00:15:42.154272 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.154152    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980"} err="failed to get container status \"7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980\": rpc error: code = NotFound desc = could not find container \"7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980\": container with ID starting with 7004b150aa79aba87b8a391689b3f06fb85594acc2e14b29b9443d9c62ad5980 not found: ID does not exist"
Jul 14 00:15:42.154272 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.154177    2550 scope.go:117] "RemoveContainer" containerID="aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"
Jul 14 00:15:42.154469 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:15:42.154447    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31\": container with ID starting with aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31 not found: ID does not exist" containerID="aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"
Jul 14 00:15:42.154510 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.154477    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31"} err="failed to get container status \"aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31\": rpc error: code = NotFound desc = could not find container \"aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31\": container with ID starting with aac2da259f9ff98077a943de191385912d6688905516454ba597078799ce0b31 not found: ID does not exist"
Jul 14 00:15:42.156066 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.156045    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"]
Jul 14 00:15:42.161684 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.161650    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-00e043-openshift-default-6ddbc855b7-2xfk6"]
Jul 14 00:15:42.777025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:42.776990    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" path="/var/lib/kubelet/pods/a5acd4f5-bad5-4ebf-b891-0445867abe97/volumes"
Jul 14 00:15:44.877708 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:44.877670    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:15:45.811298 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811252    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"]
Jul 14 00:15:45.811633 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811621    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1911f356-37a5-4f22-a24c-d0313b024e67" containerName="main"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811635    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1911f356-37a5-4f22-a24c-d0313b024e67" containerName="main"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811644    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811650    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811660    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811665    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811671    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c48fcc5-6107-44a7-aeb5-56f5ed191a97" containerName="main"
Jul 14 00:15:45.811678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811676    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c48fcc5-6107-44a7-aeb5-56f5ed191a97" containerName="main"
Jul 14 00:15:45.811878 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811728    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy"
Jul 14 00:15:45.811878 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811735    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1911f356-37a5-4f22-a24c-d0313b024e67" containerName="main"
Jul 14 00:15:45.811878 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811740    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c48fcc5-6107-44a7-aeb5-56f5ed191a97" containerName="main"
Jul 14 00:15:45.811878 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.811746    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5acd4f5-bad5-4ebf-b891-0445867abe97" containerName="istio-proxy"
Jul 14 00:15:45.814843 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.814824    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.817037 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.817015    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1ad924-openshift-default-dockercfg-47xsx\""
Jul 14 00:15:45.826147 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.826120    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"]
Jul 14 00:15:45.869060 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869019    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-data\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869060 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869066    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-credential-socket\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869093    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-istio-token\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869113    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-certs\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869143    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46l8l\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-kube-api-access-46l8l\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869232    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b64038-f866-431a-bdea-d6091af7dbc2-istio-podinfo\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b64038-f866-431a-bdea-d6091af7dbc2-istiod-ca-cert\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869291    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-socket\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.869324 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.869305    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-envoy\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970255 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970208    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46l8l\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-kube-api-access-46l8l\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970277    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b64038-f866-431a-bdea-d6091af7dbc2-istio-podinfo\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970396    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b64038-f866-431a-bdea-d6091af7dbc2-istiod-ca-cert\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970450    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-socket\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970474    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-envoy\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970548    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-data\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970593    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-credential-socket\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970635    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-istio-token\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.970729 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.970662    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-certs\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.971174 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.971020    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-socket\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.971174 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.971044    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-data\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.971264 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.971246    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b64038-f866-431a-bdea-d6091af7dbc2-istiod-ca-cert\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.971331 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.971309    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-credential-socket\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.971373 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.971317    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-certs\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.972855 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.972824    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-envoy\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.973080 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.973062    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b64038-f866-431a-bdea-d6091af7dbc2-istio-podinfo\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.978427 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.978402    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-istio-token\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:45.978562 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:45.978470    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46l8l\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-kube-api-access-46l8l\") pod \"e2e-api-b-1ad924-openshift-default-586598888c-gw646\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") " pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:46.126067 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:46.125965    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:46.280259 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:46.280220    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"]
Jul 14 00:15:46.283794 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:15:46.283750    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b64038_f866_431a_bdea_d6091af7dbc2.slice/crio-942eb6a16a20a6038bd921e4d3d2fdf7bd193c004efff72e3469a2f0c85d4561 WatchSource:0}: Error finding container 942eb6a16a20a6038bd921e4d3d2fdf7bd193c004efff72e3469a2f0c85d4561: Status 404 returned error can't find the container with id 942eb6a16a20a6038bd921e4d3d2fdf7bd193c004efff72e3469a2f0c85d4561
Jul 14 00:15:46.286073 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:46.286039    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:15:46.286156 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:46.286115    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:15:46.286156 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:46.286146    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:15:47.162756 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:47.162717    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" event={"ID":"45b64038-f866-431a-bdea-d6091af7dbc2","Type":"ContainerStarted","Data":"46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"}
Jul 14 00:15:47.163184 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:47.162774    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" event={"ID":"45b64038-f866-431a-bdea-d6091af7dbc2","Type":"ContainerStarted","Data":"942eb6a16a20a6038bd921e4d3d2fdf7bd193c004efff72e3469a2f0c85d4561"}
Jul 14 00:15:47.182853 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:47.182788    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" podStartSLOduration=2.18276958 podStartE2EDuration="2.18276958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:15:47.181264307 +0000 UTC m=+2198.979165023" watchObservedRunningTime="2026-07-14 00:15:47.18276958 +0000 UTC m=+2198.980670274"
Jul 14 00:15:48.126395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:48.126360    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:48.131780 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:48.131742    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:48.167095 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:48.167059    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:15:48.168274 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:15:48.168245    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:00.182816 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:00.182775    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:16:01.602428 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.602381    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"]
Jul 14 00:16:01.608477 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.608445    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:01.610975 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.610950    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 00:16:01.611116 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.610950    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vzxsj\""
Jul 14 00:16:01.615320 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.615284    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"]
Jul 14 00:16:01.728498 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.728453    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bdd9f1-d2d6-4782-a60f-7cb12108aac1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp\" (UID: \"88bdd9f1-d2d6-4782-a60f-7cb12108aac1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:01.829216 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.829176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bdd9f1-d2d6-4782-a60f-7cb12108aac1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp\" (UID: \"88bdd9f1-d2d6-4782-a60f-7cb12108aac1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:01.829877 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.829853    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bdd9f1-d2d6-4782-a60f-7cb12108aac1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp\" (UID: \"88bdd9f1-d2d6-4782-a60f-7cb12108aac1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:01.921810 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:01.921707    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:02.122019 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:02.121992    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"]
Jul 14 00:16:02.124951 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:16:02.124886    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bdd9f1_d2d6_4782_a60f_7cb12108aac1.slice/crio-f6b92f2756b0ad142d92b955d04e0e532b12d19cb703505320f1da7ee2bab573 WatchSource:0}: Error finding container f6b92f2756b0ad142d92b955d04e0e532b12d19cb703505320f1da7ee2bab573: Status 404 returned error can't find the container with id f6b92f2756b0ad142d92b955d04e0e532b12d19cb703505320f1da7ee2bab573
Jul 14 00:16:02.237632 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:02.237528    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp" event={"ID":"88bdd9f1-d2d6-4782-a60f-7cb12108aac1","Type":"ContainerStarted","Data":"f6b92f2756b0ad142d92b955d04e0e532b12d19cb703505320f1da7ee2bab573"}
Jul 14 00:16:03.244407 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:03.244369    2550 generic.go:356] "Generic (PLEG): container finished" podID="88bdd9f1-d2d6-4782-a60f-7cb12108aac1" containerID="76f51f5889d8c658937963a0f91e33d2fcedcced8a25ead739efdb0a70480879" exitCode=0
Jul 14 00:16:03.244847 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:03.244439    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp" event={"ID":"88bdd9f1-d2d6-4782-a60f-7cb12108aac1","Type":"ContainerDied","Data":"76f51f5889d8c658937963a0f91e33d2fcedcced8a25ead739efdb0a70480879"}
Jul 14 00:16:04.438495 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:04.438465    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:04.554768 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:04.554714    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bdd9f1-d2d6-4782-a60f-7cb12108aac1-maas-api-service-ca\") pod \"88bdd9f1-d2d6-4782-a60f-7cb12108aac1\" (UID: \"88bdd9f1-d2d6-4782-a60f-7cb12108aac1\") "
Jul 14 00:16:04.555117 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:04.555095    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88bdd9f1-d2d6-4782-a60f-7cb12108aac1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "88bdd9f1-d2d6-4782-a60f-7cb12108aac1" (UID: "88bdd9f1-d2d6-4782-a60f-7cb12108aac1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:16:04.655622 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:04.655563    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/88bdd9f1-d2d6-4782-a60f-7cb12108aac1-maas-api-service-ca\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:05.255280 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:05.255245    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp"
Jul 14 00:16:05.255280 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:05.255270    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp" event={"ID":"88bdd9f1-d2d6-4782-a60f-7cb12108aac1","Type":"ContainerDied","Data":"f6b92f2756b0ad142d92b955d04e0e532b12d19cb703505320f1da7ee2bab573"}
Jul 14 00:16:05.255518 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:05.255322    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6b92f2756b0ad142d92b955d04e0e532b12d19cb703505320f1da7ee2bab573"
Jul 14 00:16:08.272213 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:08.272095    2550 generic.go:356] "Generic (PLEG): container finished" podID="45b64038-f866-431a-bdea-d6091af7dbc2" containerID="46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836" exitCode=0
Jul 14 00:16:08.272213 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:08.272154    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" event={"ID":"45b64038-f866-431a-bdea-d6091af7dbc2","Type":"ContainerDied","Data":"46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"}
Jul 14 00:16:08.272655 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:08.272560    2550 scope.go:117] "RemoveContainer" containerID="46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"
Jul 14 00:16:08.273689 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:08.273659    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:16:08.273764 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:08.273732    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:16:08.273805 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:08.273765    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:16:09.279192 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:09.279154    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" event={"ID":"45b64038-f866-431a-bdea-d6091af7dbc2","Type":"ContainerStarted","Data":"528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd"}
Jul 14 00:16:10.127243 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:10.127206    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:10.127630 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:10.127597    2550 patch_prober.go:28] interesting pod/e2e-api-b-1ad924-openshift-default-586598888c-gw646 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 14 00:16:10.127769 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:10.127648    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 14 00:16:11.126626 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:11.126584    2550 patch_prober.go:28] interesting pod/e2e-api-b-1ad924-openshift-default-586598888c-gw646 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 14 00:16:11.127075 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:11.126646    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 14 00:16:12.126980 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:12.126936    2550 patch_prober.go:28] interesting pod/e2e-api-b-1ad924-openshift-default-586598888c-gw646 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 14 00:16:12.127484 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:12.127034    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 14 00:16:13.126449 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:13.126407    2550 patch_prober.go:28] interesting pod/e2e-api-b-1ad924-openshift-default-586598888c-gw646 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 14 00:16:13.126674 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:13.126477    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 14 00:16:14.130874 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:14.130841    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:14.131284 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:14.131156    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:14.132000 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:14.131982    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:15.979016 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:15.978974    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:16:41.085000 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:41.084893    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:16:53.475645 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:53.475602    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"]
Jul 14 00:16:53.476203 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:53.475890    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy" containerID="cri-o://528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd" gracePeriod=30
Jul 14 00:16:58.734118 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.734086    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:58.878437 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878396    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-credential-socket\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878437 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878445    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-socket\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878468    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-data\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878507    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-certs\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878541    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46l8l\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-kube-api-access-46l8l\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878558    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b64038-f866-431a-bdea-d6091af7dbc2-istio-podinfo\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878588    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-envoy\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878627    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-istio-token\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.878699 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878698    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b64038-f866-431a-bdea-d6091af7dbc2-istiod-ca-cert\") pod \"45b64038-f866-431a-bdea-d6091af7dbc2\" (UID: \"45b64038-f866-431a-bdea-d6091af7dbc2\") "
Jul 14 00:16:58.879095 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878768    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:16:58.879095 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.878892    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:16:58.879095 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.879020    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:16:58.879252 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.879202    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-credential-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.879252 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.879222    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.879252 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.879237    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-workload-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.879806 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.879743    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:16:58.880089 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.880035    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45b64038-f866-431a-bdea-d6091af7dbc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:16:58.881413 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.881361    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45b64038-f866-431a-bdea-d6091af7dbc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 00:16:58.881580 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.881555    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-kube-api-access-46l8l" (OuterVolumeSpecName: "kube-api-access-46l8l") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "kube-api-access-46l8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:16:58.881662 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.881593    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:16:58.881726 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.881704    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "45b64038-f866-431a-bdea-d6091af7dbc2" (UID: "45b64038-f866-431a-bdea-d6091af7dbc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:16:58.979983 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.979942    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-envoy\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.979983 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.979976    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-istio-token\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.979983 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.979985    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45b64038-f866-431a-bdea-d6091af7dbc2-istiod-ca-cert\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.979983 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.979993    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45b64038-f866-431a-bdea-d6091af7dbc2-istio-data\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.980281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.980001    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46l8l\" (UniqueName: \"kubernetes.io/projected/45b64038-f866-431a-bdea-d6091af7dbc2-kube-api-access-46l8l\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:58.980281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:58.980010    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45b64038-f866-431a-bdea-d6091af7dbc2-istio-podinfo\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:16:59.538386 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.538351    2550 generic.go:356] "Generic (PLEG): container finished" podID="45b64038-f866-431a-bdea-d6091af7dbc2" containerID="528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd" exitCode=0
Jul 14 00:16:59.538599 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.538418    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"
Jul 14 00:16:59.538599 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.538436    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" event={"ID":"45b64038-f866-431a-bdea-d6091af7dbc2","Type":"ContainerDied","Data":"528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd"}
Jul 14 00:16:59.538599 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.538513    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646" event={"ID":"45b64038-f866-431a-bdea-d6091af7dbc2","Type":"ContainerDied","Data":"942eb6a16a20a6038bd921e4d3d2fdf7bd193c004efff72e3469a2f0c85d4561"}
Jul 14 00:16:59.538599 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.538548    2550 scope.go:117] "RemoveContainer" containerID="528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd"
Jul 14 00:16:59.548709 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.548579    2550 scope.go:117] "RemoveContainer" containerID="46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"
Jul 14 00:16:59.558532 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.558507    2550 scope.go:117] "RemoveContainer" containerID="528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd"
Jul 14 00:16:59.558834 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:16:59.558810    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd\": container with ID starting with 528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd not found: ID does not exist" containerID="528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd"
Jul 14 00:16:59.558882 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.558847    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd"} err="failed to get container status \"528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd\": rpc error: code = NotFound desc = could not find container \"528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd\": container with ID starting with 528855b712ef11e129ec2610d9d24aa2454154e34ffc27a7ea634d16d58961fd not found: ID does not exist"
Jul 14 00:16:59.558882 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.558874    2550 scope.go:117] "RemoveContainer" containerID="46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"
Jul 14 00:16:59.559159 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:16:59.559136    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836\": container with ID starting with 46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836 not found: ID does not exist" containerID="46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"
Jul 14 00:16:59.559208 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.559171    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836"} err="failed to get container status \"46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836\": rpc error: code = NotFound desc = could not find container \"46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836\": container with ID starting with 46e66b6eaf3a92ca874c9beb3059f866cff84c935eb3521901d0f78616399836 not found: ID does not exist"
Jul 14 00:16:59.563217 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.563178    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"]
Jul 14 00:16:59.567412 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:16:59.567387    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1ad924-openshift-default-586598888c-gw646"]
Jul 14 00:17:00.777120 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:00.777088    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" path="/var/lib/kubelet/pods/45b64038-f866-431a-bdea-d6091af7dbc2/volumes"
Jul 14 00:17:02.098127 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.098087    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"]
Jul 14 00:17:02.098950 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.098923    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy"
Jul 14 00:17:02.098950 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.098951    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy"
Jul 14 00:17:02.099228 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.098973    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy"
Jul 14 00:17:02.099228 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.098983    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy"
Jul 14 00:17:02.099228 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.099023    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88bdd9f1-d2d6-4782-a60f-7cb12108aac1" containerName="revoke-keys"
Jul 14 00:17:02.099228 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.099033    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="88bdd9f1-d2d6-4782-a60f-7cb12108aac1" containerName="revoke-keys"
Jul 14 00:17:02.099420 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.099233    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="88bdd9f1-d2d6-4782-a60f-7cb12108aac1" containerName="revoke-keys"
Jul 14 00:17:02.099420 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.099259    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy"
Jul 14 00:17:02.099420 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.099275    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b64038-f866-431a-bdea-d6091af7dbc2" containerName="istio-proxy"
Jul 14 00:17:02.104787 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.104762    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.107712 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.107692    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-09117f-openshift-default-dockercfg-699gn\""
Jul 14 00:17:02.136011 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.135975    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"]
Jul 14 00:17:02.205207 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205168    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-credential-socket\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205421 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205254    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-certs\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205421 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205281    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5ba35b9-5280-42ba-974b-80823becdceb-istiod-ca-cert\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205421 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205295    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lqj4\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-kube-api-access-4lqj4\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205421 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5ba35b9-5280-42ba-974b-80823becdceb-istio-podinfo\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205562 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205422    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-socket\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205562 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205456    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-data\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205562 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205486    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-istio-token\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.205562 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.205514    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-envoy\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.306829 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.306790    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-credential-socket\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.306910    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-certs\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.306947    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5ba35b9-5280-42ba-974b-80823becdceb-istiod-ca-cert\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.306972    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lqj4\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-kube-api-access-4lqj4\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307052    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5ba35b9-5280-42ba-974b-80823becdceb-istio-podinfo\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307100 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307089    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-socket\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307110    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-data\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307179    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-istio-token\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307291    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-envoy\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.307880 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307855    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-socket\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.308049 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.308027    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-certs\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.308162 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.307948    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-data\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.308307 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.308144    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-credential-socket\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.308443 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.308418    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5ba35b9-5280-42ba-974b-80823becdceb-istiod-ca-cert\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.309725 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.309701    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:17:02.309967 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.309946    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-envoy\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.310195 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.310160    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5ba35b9-5280-42ba-974b-80823becdceb-istio-podinfo\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.315145 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.315124    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lqj4\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-kube-api-access-4lqj4\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.315145 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.315135    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-istio-token\") pod \"e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") " pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.415347 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.415252    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:02.560798 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.560767    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"]
Jul 14 00:17:02.563390 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:17:02.563362    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ba35b9_5280_42ba_974b_80823becdceb.slice/crio-01eb7fa3fb278d0d88814410c0582d4ed8eeb8f8e0f80ca7941c1b2b152bbc5b WatchSource:0}: Error finding container 01eb7fa3fb278d0d88814410c0582d4ed8eeb8f8e0f80ca7941c1b2b152bbc5b: Status 404 returned error can't find the container with id 01eb7fa3fb278d0d88814410c0582d4ed8eeb8f8e0f80ca7941c1b2b152bbc5b
Jul 14 00:17:02.565646 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.565611    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:17:02.565724 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.565687    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:17:02.565724 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:02.565715    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:17:03.561292 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:03.561257    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" event={"ID":"b5ba35b9-5280-42ba-974b-80823becdceb","Type":"ContainerStarted","Data":"a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"}
Jul 14 00:17:03.561790 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:03.561310    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" event={"ID":"b5ba35b9-5280-42ba-974b-80823becdceb","Type":"ContainerStarted","Data":"01eb7fa3fb278d0d88814410c0582d4ed8eeb8f8e0f80ca7941c1b2b152bbc5b"}
Jul 14 00:17:03.586701 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:03.586650    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podStartSLOduration=1.586634712 podStartE2EDuration="1.586634712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:17:03.585406517 +0000 UTC m=+2275.383307226" watchObservedRunningTime="2026-07-14 00:17:03.586634712 +0000 UTC m=+2275.384535409"
Jul 14 00:17:04.415430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:04.415392    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:04.420802 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:04.420771    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:04.565916 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:04.565874    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:04.567021 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:04.567000    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:11.885616 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:11.885577    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:17:19.567333 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.567289    2550 patch_prober.go:28] interesting pod/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 14 00:17:19.567754 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.567383    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 14 00:17:19.643620 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.643525    2550 generic.go:356] "Generic (PLEG): container finished" podID="b5ba35b9-5280-42ba-974b-80823becdceb" containerID="a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4" exitCode=0
Jul 14 00:17:19.643620 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.643587    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" event={"ID":"b5ba35b9-5280-42ba-974b-80823becdceb","Type":"ContainerDied","Data":"a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"}
Jul 14 00:17:19.644225 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.644206    2550 scope.go:117] "RemoveContainer" containerID="a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"
Jul 14 00:17:19.645308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.645276    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:17:19.645378 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.645355    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:17:19.645427 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:19.645397    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892172Ki","pods":"250"}
Jul 14 00:17:20.651808 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:20.651762    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" event={"ID":"b5ba35b9-5280-42ba-974b-80823becdceb","Type":"ContainerStarted","Data":"d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56"}
Jul 14 00:17:21.415479 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:21.415429    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:21.415717 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:21.415690    2550 patch_prober.go:28] interesting pod/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 14 00:17:21.415794 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:21.415770    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 14 00:17:22.416357 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:22.416308    2550 patch_prober.go:28] interesting pod/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 14 00:17:22.416773 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:22.416379    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 14 00:17:23.415839 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:23.415793    2550 patch_prober.go:28] interesting pod/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 14 00:17:23.416063 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:23.415855    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 14 00:17:24.416752 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:24.416675    2550 patch_prober.go:28] interesting pod/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 14 00:17:24.417673 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:24.416792    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 14 00:17:25.420619 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:25.420585    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:25.421096 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:25.420935    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:25.421839 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:25.421818    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:17:37.847870 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.847820    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"]
Jul 14 00:17:37.850650 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.850625    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:37.854049 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.854023    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"openshift-service-ca.crt\""
Jul 14 00:17:37.854176 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.854023    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"auth-test-model-7e917f-kserve-self-signed-certs\""
Jul 14 00:17:37.854176 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.854074    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"default-dockercfg-xdrvx\""
Jul 14 00:17:37.854176 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.854027    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"kube-root-ca.crt\""
Jul 14 00:17:37.861531 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.861504    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"]
Jul 14 00:17:37.946327 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.946292    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mbzk\" (UniqueName: \"kubernetes.io/projected/665e8b07-d353-4db6-9d74-2de32e7a0bdc-kube-api-access-4mbzk\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:37.946532 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.946342    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-model-cache\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:37.946532 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.946461    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tls-certs\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:37.946652 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.946531    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tmp-dir\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:37.946652 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.946584    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-home\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:37.946739 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:37.946663    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-dshm\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048157 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048093    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mbzk\" (UniqueName: \"kubernetes.io/projected/665e8b07-d353-4db6-9d74-2de32e7a0bdc-kube-api-access-4mbzk\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048385 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048184    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-model-cache\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048385 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048315    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tls-certs\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048385 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048359    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tmp-dir\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048558 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048406    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-home\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048558 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048440    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-dshm\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048665 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048618    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-model-cache\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048712 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048685    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tmp-dir\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.048808 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.048787    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-home\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.050835 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.050808    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-dshm\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.051143 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.051124    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tls-certs\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.063290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.063255    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mbzk\" (UniqueName: \"kubernetes.io/projected/665e8b07-d353-4db6-9d74-2de32e7a0bdc-kube-api-access-4mbzk\") pod \"auth-test-model-7e917f-kserve-5466446ddc-mjs9f\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") " pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.162775 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.162615    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:38.309849 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.309809    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"]
Jul 14 00:17:38.313500 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:17:38.313464    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod665e8b07_d353_4db6_9d74_2de32e7a0bdc.slice/crio-7b6d34b75f281f81def88d33571bcc56db87e66533f07c63b1cfa4e47c07d49c WatchSource:0}: Error finding container 7b6d34b75f281f81def88d33571bcc56db87e66533f07c63b1cfa4e47c07d49c: Status 404 returned error can't find the container with id 7b6d34b75f281f81def88d33571bcc56db87e66533f07c63b1cfa4e47c07d49c
Jul 14 00:17:38.740973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.740932    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f" event={"ID":"665e8b07-d353-4db6-9d74-2de32e7a0bdc","Type":"ContainerStarted","Data":"5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5"}
Jul 14 00:17:38.741162 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.740983    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f" event={"ID":"665e8b07-d353-4db6-9d74-2de32e7a0bdc","Type":"ContainerStarted","Data":"7b6d34b75f281f81def88d33571bcc56db87e66533f07c63b1cfa4e47c07d49c"}
Jul 14 00:17:38.758441 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:38.758384    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f" podStartSLOduration=1.7583689900000001 podStartE2EDuration="1.75836899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:17:38.756775274 +0000 UTC m=+2310.554675974" watchObservedRunningTime="2026-07-14 00:17:38.75836899 +0000 UTC m=+2310.556269689"
Jul 14 00:17:40.682717 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:40.682667    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:17:48.163572 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:48.163532    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:48.164130 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:48.163680    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:48.176236 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:48.176206    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:17:48.801487 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:17:48.801456    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:18:03.646889 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.646659    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"]
Jul 14 00:18:03.651463 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.651435    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.654185 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.654145    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"kube-root-ca.crt\""
Jul 14 00:18:03.655361 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.655335    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"openshift-service-ca.crt\""
Jul 14 00:18:03.655510 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.655351    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"auth-test-model-09117f-kserve-self-signed-certs\""
Jul 14 00:18:03.655510 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.655357    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"default-dockercfg-qwpzb\""
Jul 14 00:18:03.661589 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.661561    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"]
Jul 14 00:18:03.797454 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.797393    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tmp-dir\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.797454 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.797444    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tls-certs\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.797454 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.797465    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-dshm\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.797789 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.797551    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-home\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.797789 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.797591    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5fbl\" (UniqueName: \"kubernetes.io/projected/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-kube-api-access-v5fbl\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.797789 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.797657    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-model-cache\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898475 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898377    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-model-cache\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898475 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898456    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tmp-dir\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898475 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898473    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tls-certs\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898776 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898491    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-dshm\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898776 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898522    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-home\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898776 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898545    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5fbl\" (UniqueName: \"kubernetes.io/projected/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-kube-api-access-v5fbl\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898990 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898860    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-model-cache\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898990 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898947    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tmp-dir\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.898990 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.898957    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-home\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.901046 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.901026    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-dshm\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.901320 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.901302    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tls-certs\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.906662 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.906641    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5fbl\" (UniqueName: \"kubernetes.io/projected/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-kube-api-access-v5fbl\") pod \"auth-test-model-09117f-kserve-8875d848c-wm9ln\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") " pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:03.964360 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:03.964321    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:04.115225 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:04.115193    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"]
Jul 14 00:18:04.117443 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:18:04.117398    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd58a8ca2_3f9a_40bf_a112_0a36039be5fe.slice/crio-d4d16453480ccf96c43dcc9671030558dc3df54d16a239a126132df8cfd05dc9 WatchSource:0}: Error finding container d4d16453480ccf96c43dcc9671030558dc3df54d16a239a126132df8cfd05dc9: Status 404 returned error can't find the container with id d4d16453480ccf96c43dcc9671030558dc3df54d16a239a126132df8cfd05dc9
Jul 14 00:18:04.870327 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:04.870276    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln" event={"ID":"d58a8ca2-3f9a-40bf-a112-0a36039be5fe","Type":"ContainerStarted","Data":"e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b"}
Jul 14 00:18:04.870327 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:04.870334    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln" event={"ID":"d58a8ca2-3f9a-40bf-a112-0a36039be5fe","Type":"ContainerStarted","Data":"d4d16453480ccf96c43dcc9671030558dc3df54d16a239a126132df8cfd05dc9"}
Jul 14 00:18:04.887563 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:04.887474    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln" podStartSLOduration=1.887451751 podStartE2EDuration="1.887451751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:18:04.886359844 +0000 UTC m=+2336.684260539" watchObservedRunningTime="2026-07-14 00:18:04.887451751 +0000 UTC m=+2336.685352451"
Jul 14 00:18:07.182107 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:07.182064    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:18:13.964496 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:13.964448    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:13.964496 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:13.964504    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:13.977561 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:13.977530    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:14.932196 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:14.932135    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:18:22.179011 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:22.178967    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:18:40.281060 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:40.280999    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:18:57.585318 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:18:57.585278    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:19:08.848025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:08.847992    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:19:08.856658 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:08.856631    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:19:08.856845 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:08.856793    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:19:08.864913 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:08.864876    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:19:13.979220 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:13.979181    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:19:31.583890 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:31.583802    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:19:49.483954 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:19:49.483890    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:20:02.653649 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:02.653613    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"]
Jul 14 00:20:03.726783 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:03.726741    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"]
Jul 14 00:20:07.243014 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.242980    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"]
Jul 14 00:20:07.246637 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.246615    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.249088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.249067    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"sub-test-model-7e917f-kserve-self-signed-certs\""
Jul 14 00:20:07.257380 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.257355    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"]
Jul 14 00:20:07.302414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.302371    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dptxf\" (UniqueName: \"kubernetes.io/projected/270d5d29-e38c-4226-9b97-cb65d9b60143-kube-api-access-dptxf\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.302414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.302415    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270d5d29-e38c-4226-9b97-cb65d9b60143-tls-certs\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.302632 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.302438    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-dshm\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.302632 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.302516    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-home\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.302632 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.302565    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-model-cache\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.302632 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.302582    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-tmp-dir\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.403694 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.403654    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dptxf\" (UniqueName: \"kubernetes.io/projected/270d5d29-e38c-4226-9b97-cb65d9b60143-kube-api-access-dptxf\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.403694 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.403698    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270d5d29-e38c-4226-9b97-cb65d9b60143-tls-certs\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.403951 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.403720    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-dshm\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.403951 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.403762    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-home\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.403951 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.403795    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-model-cache\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.403951 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.403810    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-tmp-dir\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.404302 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.404277    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-tmp-dir\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.404378 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.404304    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-model-cache\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.404378 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.404336    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-home\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.406319 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.406296    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-dshm\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.406495 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.406476    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270d5d29-e38c-4226-9b97-cb65d9b60143-tls-certs\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.411722 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.411695    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dptxf\" (UniqueName: \"kubernetes.io/projected/270d5d29-e38c-4226-9b97-cb65d9b60143-kube-api-access-dptxf\") pod \"sub-test-model-7e917f-kserve-695b786bf7-j8l4s\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") " pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.557432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.557385    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:07.683372 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.683332    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:20:07.701180 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:07.701155    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"]
Jul 14 00:20:07.704047 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:20:07.704016    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod270d5d29_e38c_4226_9b97_cb65d9b60143.slice/crio-f489b97289b62c6b80c30a15bcfb690ea7726504c87f1405b66ae25d4dd88b39 WatchSource:0}: Error finding container f489b97289b62c6b80c30a15bcfb690ea7726504c87f1405b66ae25d4dd88b39: Status 404 returned error can't find the container with id f489b97289b62c6b80c30a15bcfb690ea7726504c87f1405b66ae25d4dd88b39
Jul 14 00:20:08.495693 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:08.495650    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s" event={"ID":"270d5d29-e38c-4226-9b97-cb65d9b60143","Type":"ContainerStarted","Data":"8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1"}
Jul 14 00:20:08.496178 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:08.495705    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s" event={"ID":"270d5d29-e38c-4226-9b97-cb65d9b60143","Type":"ContainerStarted","Data":"f489b97289b62c6b80c30a15bcfb690ea7726504c87f1405b66ae25d4dd88b39"}
Jul 14 00:20:08.512853 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:08.512792    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s" podStartSLOduration=1.512771192 podStartE2EDuration="1.512771192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:20:08.511273728 +0000 UTC m=+2460.309174427" watchObservedRunningTime="2026-07-14 00:20:08.512771192 +0000 UTC m=+2460.310671868"
Jul 14 00:20:17.558295 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.558248    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:17.558870 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.558851    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:17.588474 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.588431    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:17.674917 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.674856    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f" podUID="665e8b07-d353-4db6-9d74-2de32e7a0bdc" containerName="main" containerID="cri-o://5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5" gracePeriod=45
Jul 14 00:20:17.958435 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.958405    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:20:17.995995 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.995952    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tls-certs\") pod \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") "
Jul 14 00:20:17.995995 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996002    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-model-cache\") pod \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") "
Jul 14 00:20:17.996264 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996042    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mbzk\" (UniqueName: \"kubernetes.io/projected/665e8b07-d353-4db6-9d74-2de32e7a0bdc-kube-api-access-4mbzk\") pod \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") "
Jul 14 00:20:17.996264 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996120    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-dshm\") pod \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") "
Jul 14 00:20:17.996264 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996185    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tmp-dir\") pod \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") "
Jul 14 00:20:17.996264 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996260    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-home\") pod \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\" (UID: \"665e8b07-d353-4db6-9d74-2de32e7a0bdc\") "
Jul 14 00:20:17.996549 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996297    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-model-cache" (OuterVolumeSpecName: "model-cache") pod "665e8b07-d353-4db6-9d74-2de32e7a0bdc" (UID: "665e8b07-d353-4db6-9d74-2de32e7a0bdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:17.996549 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996518    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-home" (OuterVolumeSpecName: "home") pod "665e8b07-d353-4db6-9d74-2de32e7a0bdc" (UID: "665e8b07-d353-4db6-9d74-2de32e7a0bdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:17.996665 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996567    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:17.996723 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.996696    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "665e8b07-d353-4db6-9d74-2de32e7a0bdc" (UID: "665e8b07-d353-4db6-9d74-2de32e7a0bdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:17.998656 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.998624    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "665e8b07-d353-4db6-9d74-2de32e7a0bdc" (UID: "665e8b07-d353-4db6-9d74-2de32e7a0bdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:20:17.999079 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.999056    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-dshm" (OuterVolumeSpecName: "dshm") pod "665e8b07-d353-4db6-9d74-2de32e7a0bdc" (UID: "665e8b07-d353-4db6-9d74-2de32e7a0bdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:17.999207 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:17.999082    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/665e8b07-d353-4db6-9d74-2de32e7a0bdc-kube-api-access-4mbzk" (OuterVolumeSpecName: "kube-api-access-4mbzk") pod "665e8b07-d353-4db6-9d74-2de32e7a0bdc" (UID: "665e8b07-d353-4db6-9d74-2de32e7a0bdc"). InnerVolumeSpecName "kube-api-access-4mbzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:20:18.097941 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.097879    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:18.097941 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.097932    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:18.097941 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.097946    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mbzk\" (UniqueName: \"kubernetes.io/projected/665e8b07-d353-4db6-9d74-2de32e7a0bdc-kube-api-access-4mbzk\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:18.097941 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.097955    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:18.098322 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.097964    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/665e8b07-d353-4db6-9d74-2de32e7a0bdc-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:18.547096 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.547057    2550 generic.go:356] "Generic (PLEG): container finished" podID="665e8b07-d353-4db6-9d74-2de32e7a0bdc" containerID="5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5" exitCode=0
Jul 14 00:20:18.547294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.547108    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"
Jul 14 00:20:18.547294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.547142    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f" event={"ID":"665e8b07-d353-4db6-9d74-2de32e7a0bdc","Type":"ContainerDied","Data":"5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5"}
Jul 14 00:20:18.547294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.547191    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f" event={"ID":"665e8b07-d353-4db6-9d74-2de32e7a0bdc","Type":"ContainerDied","Data":"7b6d34b75f281f81def88d33571bcc56db87e66533f07c63b1cfa4e47c07d49c"}
Jul 14 00:20:18.547294 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.547212    2550 scope.go:117] "RemoveContainer" containerID="5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5"
Jul 14 00:20:18.558354 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.558330    2550 scope.go:117] "RemoveContainer" containerID="5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5"
Jul 14 00:20:18.558707 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:20:18.558685    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5\": container with ID starting with 5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5 not found: ID does not exist" containerID="5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5"
Jul 14 00:20:18.558764 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.558717    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5"} err="failed to get container status \"5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5\": rpc error: code = NotFound desc = could not find container \"5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5\": container with ID starting with 5087f135a4138dd0d3eb140a7eb0d7bdf8bbffaeab010b7cec40b944cb8608c5 not found: ID does not exist"
Jul 14 00:20:18.561652 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.561627    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:20:18.577182 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.577142    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"]
Jul 14 00:20:18.580431 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.580400    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/auth-test-model-7e917f-kserve-5466446ddc-mjs9f"]
Jul 14 00:20:18.748262 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.748219    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln" podUID="d58a8ca2-3f9a-40bf-a112-0a36039be5fe" containerName="main" containerID="cri-o://e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b" gracePeriod=45
Jul 14 00:20:18.780429 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:18.779354    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="665e8b07-d353-4db6-9d74-2de32e7a0bdc" path="/var/lib/kubelet/pods/665e8b07-d353-4db6-9d74-2de32e7a0bdc/volumes"
Jul 14 00:20:19.011310 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.011282    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:20:19.106443 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106405    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tmp-dir\") pod \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") "
Jul 14 00:20:19.106636 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106469    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-dshm\") pod \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") "
Jul 14 00:20:19.106636 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106550    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tls-certs\") pod \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") "
Jul 14 00:20:19.106636 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106581    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-home\") pod \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") "
Jul 14 00:20:19.106636 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106607    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5fbl\" (UniqueName: \"kubernetes.io/projected/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-kube-api-access-v5fbl\") pod \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") "
Jul 14 00:20:19.106836 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106706    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-model-cache\") pod \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\" (UID: \"d58a8ca2-3f9a-40bf-a112-0a36039be5fe\") "
Jul 14 00:20:19.106836 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106743    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d58a8ca2-3f9a-40bf-a112-0a36039be5fe" (UID: "d58a8ca2-3f9a-40bf-a112-0a36039be5fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:19.106993 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.106862    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-home" (OuterVolumeSpecName: "home") pod "d58a8ca2-3f9a-40bf-a112-0a36039be5fe" (UID: "d58a8ca2-3f9a-40bf-a112-0a36039be5fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:19.107058 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.107017    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:19.107058 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.107035    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:19.107150 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.107072    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "d58a8ca2-3f9a-40bf-a112-0a36039be5fe" (UID: "d58a8ca2-3f9a-40bf-a112-0a36039be5fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:19.109011 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.108977    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d58a8ca2-3f9a-40bf-a112-0a36039be5fe" (UID: "d58a8ca2-3f9a-40bf-a112-0a36039be5fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:20:19.109011 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.108990    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-kube-api-access-v5fbl" (OuterVolumeSpecName: "kube-api-access-v5fbl") pod "d58a8ca2-3f9a-40bf-a112-0a36039be5fe" (UID: "d58a8ca2-3f9a-40bf-a112-0a36039be5fe"). InnerVolumeSpecName "kube-api-access-v5fbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:20:19.109011 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.108967    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-dshm" (OuterVolumeSpecName: "dshm") pod "d58a8ca2-3f9a-40bf-a112-0a36039be5fe" (UID: "d58a8ca2-3f9a-40bf-a112-0a36039be5fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:20:19.207569 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.207479    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:19.207569 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.207511    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:19.207569 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.207520    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:19.207569 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.207530    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5fbl\" (UniqueName: \"kubernetes.io/projected/d58a8ca2-3f9a-40bf-a112-0a36039be5fe-kube-api-access-v5fbl\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:20:19.553530 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.553496    2550 generic.go:356] "Generic (PLEG): container finished" podID="d58a8ca2-3f9a-40bf-a112-0a36039be5fe" containerID="e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b" exitCode=0
Jul 14 00:20:19.553707 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.553541    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"
Jul 14 00:20:19.553707 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.553588    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln" event={"ID":"d58a8ca2-3f9a-40bf-a112-0a36039be5fe","Type":"ContainerDied","Data":"e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b"}
Jul 14 00:20:19.553707 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.553636    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln" event={"ID":"d58a8ca2-3f9a-40bf-a112-0a36039be5fe","Type":"ContainerDied","Data":"d4d16453480ccf96c43dcc9671030558dc3df54d16a239a126132df8cfd05dc9"}
Jul 14 00:20:19.553707 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.553662    2550 scope.go:117] "RemoveContainer" containerID="e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b"
Jul 14 00:20:19.563110 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.563087    2550 scope.go:117] "RemoveContainer" containerID="e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b"
Jul 14 00:20:19.563422 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:20:19.563402    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b\": container with ID starting with e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b not found: ID does not exist" containerID="e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b"
Jul 14 00:20:19.563475 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.563430    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b"} err="failed to get container status \"e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b\": rpc error: code = NotFound desc = could not find container \"e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b\": container with ID starting with e5579344931a034c0b04472b78f32a4f8a52bc3cae682eab6eb48e396a3ec97b not found: ID does not exist"
Jul 14 00:20:19.588465 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.588431    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"]
Jul 14 00:20:19.590242 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:19.590221    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/auth-test-model-09117f-kserve-8875d848c-wm9ln"]
Jul 14 00:20:20.776751 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:20.776713    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58a8ca2-3f9a-40bf-a112-0a36039be5fe" path="/var/lib/kubelet/pods/d58a8ca2-3f9a-40bf-a112-0a36039be5fe/volumes"
Jul 14 00:20:23.078499 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:23.078455    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:20:48.745939 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.745884    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"]
Jul 14 00:20:48.746432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.746263    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="665e8b07-d353-4db6-9d74-2de32e7a0bdc" containerName="main"
Jul 14 00:20:48.746432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.746275    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="665e8b07-d353-4db6-9d74-2de32e7a0bdc" containerName="main"
Jul 14 00:20:48.746432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.746292    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58a8ca2-3f9a-40bf-a112-0a36039be5fe" containerName="main"
Jul 14 00:20:48.746432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.746297    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58a8ca2-3f9a-40bf-a112-0a36039be5fe" containerName="main"
Jul 14 00:20:48.746432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.746349    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="665e8b07-d353-4db6-9d74-2de32e7a0bdc" containerName="main"
Jul 14 00:20:48.746432 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.746360    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58a8ca2-3f9a-40bf-a112-0a36039be5fe" containerName="main"
Jul 14 00:20:48.749210 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.749191    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.751550 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.751526    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"openshift-service-ca.crt\""
Jul 14 00:20:48.752397 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.752377    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"kube-root-ca.crt\""
Jul 14 00:20:48.752504 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.752402    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"sub-test-model-09117f-kserve-self-signed-certs\""
Jul 14 00:20:48.752504 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.752443    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"default-dockercfg-qwpzb\""
Jul 14 00:20:48.759106 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.759083    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"]
Jul 14 00:20:48.894140 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.894102    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tmp-dir\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.894140 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.894142    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-dshm\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.894371 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.894166    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-model-cache\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.894371 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.894279    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48ns2\" (UniqueName: \"kubernetes.io/projected/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-kube-api-access-48ns2\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.894371 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.894353    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-home\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.894475 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.894375    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tls-certs\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.995455 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995414    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-home\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.995455 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995454    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tls-certs\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.995721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995508    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tmp-dir\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.995721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995525    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-dshm\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.995721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995541    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-model-cache\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.995721 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995607    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48ns2\" (UniqueName: \"kubernetes.io/projected/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-kube-api-access-48ns2\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.996007 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995945    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-home\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.996007 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.995965    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tmp-dir\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.996144 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.996027    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-model-cache\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.998312 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.998287    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tls-certs\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:48.998411 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:48.998321    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-dshm\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:49.004746 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:49.004720    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48ns2\" (UniqueName: \"kubernetes.io/projected/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-kube-api-access-48ns2\") pod \"sub-test-model-09117f-kserve-565c6cc6ff-psfrm\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") " pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:49.059948 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:49.059881    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:49.204800 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:49.204775    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"]
Jul 14 00:20:49.206787 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:20:49.206754    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4ffbc43_30c6_49cc_9db5_9af5106e3dda.slice/crio-e2cecbad2a198151ebe7803389959df0dc827b75bc96a1b29305594398452873 WatchSource:0}: Error finding container e2cecbad2a198151ebe7803389959df0dc827b75bc96a1b29305594398452873: Status 404 returned error can't find the container with id e2cecbad2a198151ebe7803389959df0dc827b75bc96a1b29305594398452873
Jul 14 00:20:49.709140 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:49.709100    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm" event={"ID":"f4ffbc43-30c6-49cc-9db5-9af5106e3dda","Type":"ContainerStarted","Data":"b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f"}
Jul 14 00:20:49.709321 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:49.709152    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm" event={"ID":"f4ffbc43-30c6-49cc-9db5-9af5106e3dda","Type":"ContainerStarted","Data":"e2cecbad2a198151ebe7803389959df0dc827b75bc96a1b29305594398452873"}
Jul 14 00:20:49.727695 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:49.727637    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm" podStartSLOduration=1.727621848 podStartE2EDuration="1.727621848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:20:49.725009813 +0000 UTC m=+2501.522910530" watchObservedRunningTime="2026-07-14 00:20:49.727621848 +0000 UTC m=+2501.525522547"
Jul 14 00:20:53.074675 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:53.074613    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:20:59.060642 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:59.060605    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:59.061167 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:59.060656    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:59.073414 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:59.073372    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:20:59.770135 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:20:59.770052    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:21:14.185331 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:14.185296    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:21:33.478651 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:33.478610    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:21:38.354915 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:38.354868    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"]
Jul 14 00:21:39.227455 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:39.227412    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"]
Jul 14 00:21:50.779292 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:50.779252    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:21:53.378771 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.378690    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s" podUID="270d5d29-e38c-4226-9b97-cb65d9b60143" containerName="main" containerID="cri-o://8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1" gracePeriod=45
Jul 14 00:21:53.640801 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.640727    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:21:53.713665 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713623    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-home\") pod \"270d5d29-e38c-4226-9b97-cb65d9b60143\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") "
Jul 14 00:21:53.713665 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713671    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-tmp-dir\") pod \"270d5d29-e38c-4226-9b97-cb65d9b60143\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") "
Jul 14 00:21:53.713935 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713702    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-dshm\") pod \"270d5d29-e38c-4226-9b97-cb65d9b60143\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") "
Jul 14 00:21:53.713935 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713765    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dptxf\" (UniqueName: \"kubernetes.io/projected/270d5d29-e38c-4226-9b97-cb65d9b60143-kube-api-access-dptxf\") pod \"270d5d29-e38c-4226-9b97-cb65d9b60143\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") "
Jul 14 00:21:53.713935 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713805    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-model-cache\") pod \"270d5d29-e38c-4226-9b97-cb65d9b60143\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") "
Jul 14 00:21:53.713935 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713830    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270d5d29-e38c-4226-9b97-cb65d9b60143-tls-certs\") pod \"270d5d29-e38c-4226-9b97-cb65d9b60143\" (UID: \"270d5d29-e38c-4226-9b97-cb65d9b60143\") "
Jul 14 00:21:53.714156 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.713993    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "270d5d29-e38c-4226-9b97-cb65d9b60143" (UID: "270d5d29-e38c-4226-9b97-cb65d9b60143"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:53.714156 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.714013    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-home" (OuterVolumeSpecName: "home") pod "270d5d29-e38c-4226-9b97-cb65d9b60143" (UID: "270d5d29-e38c-4226-9b97-cb65d9b60143"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:53.714259 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.714169    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-model-cache" (OuterVolumeSpecName: "model-cache") pod "270d5d29-e38c-4226-9b97-cb65d9b60143" (UID: "270d5d29-e38c-4226-9b97-cb65d9b60143"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:53.714319 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.714258    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:53.714319 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.714279    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:53.714319 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.714295    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:53.716094 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.716070    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-dshm" (OuterVolumeSpecName: "dshm") pod "270d5d29-e38c-4226-9b97-cb65d9b60143" (UID: "270d5d29-e38c-4226-9b97-cb65d9b60143"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:53.716206 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.716144    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/270d5d29-e38c-4226-9b97-cb65d9b60143-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "270d5d29-e38c-4226-9b97-cb65d9b60143" (UID: "270d5d29-e38c-4226-9b97-cb65d9b60143"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:21:53.716253 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.716214    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/270d5d29-e38c-4226-9b97-cb65d9b60143-kube-api-access-dptxf" (OuterVolumeSpecName: "kube-api-access-dptxf") pod "270d5d29-e38c-4226-9b97-cb65d9b60143" (UID: "270d5d29-e38c-4226-9b97-cb65d9b60143"). InnerVolumeSpecName "kube-api-access-dptxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:21:53.814794 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.814748    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270d5d29-e38c-4226-9b97-cb65d9b60143-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:53.814794 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.814782    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dptxf\" (UniqueName: \"kubernetes.io/projected/270d5d29-e38c-4226-9b97-cb65d9b60143-kube-api-access-dptxf\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:53.814794 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:53.814799    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270d5d29-e38c-4226-9b97-cb65d9b60143-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.025886 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.025781    2550 generic.go:356] "Generic (PLEG): container finished" podID="270d5d29-e38c-4226-9b97-cb65d9b60143" containerID="8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1" exitCode=0
Jul 14 00:21:54.025886 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.025828    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"
Jul 14 00:21:54.025886 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.025846    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s" event={"ID":"270d5d29-e38c-4226-9b97-cb65d9b60143","Type":"ContainerDied","Data":"8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1"}
Jul 14 00:21:54.025886 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.025882    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s" event={"ID":"270d5d29-e38c-4226-9b97-cb65d9b60143","Type":"ContainerDied","Data":"f489b97289b62c6b80c30a15bcfb690ea7726504c87f1405b66ae25d4dd88b39"}
Jul 14 00:21:54.026257 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.025917    2550 scope.go:117] "RemoveContainer" containerID="8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1"
Jul 14 00:21:54.035692 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.035672    2550 scope.go:117] "RemoveContainer" containerID="8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1"
Jul 14 00:21:54.036031 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:21:54.035989    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1\": container with ID starting with 8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1 not found: ID does not exist" containerID="8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1"
Jul 14 00:21:54.036111 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.036044    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1"} err="failed to get container status \"8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1\": rpc error: code = NotFound desc = could not find container \"8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1\": container with ID starting with 8dc93b3fe80a8c055f6dfa7f5641c8925b9df5defd45731d80607687b8ba61d1 not found: ID does not exist"
Jul 14 00:21:54.045632 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.045600    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"]
Jul 14 00:21:54.049800 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.049774    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/sub-test-model-7e917f-kserve-695b786bf7-j8l4s"]
Jul 14 00:21:54.249651 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.249573    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm" podUID="f4ffbc43-30c6-49cc-9db5-9af5106e3dda" containerName="main" containerID="cri-o://b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f" gracePeriod=45
Jul 14 00:21:54.519734 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.519702    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:21:54.621005 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.620884    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-model-cache\") pod \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") "
Jul 14 00:21:54.621005 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.620953    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48ns2\" (UniqueName: \"kubernetes.io/projected/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-kube-api-access-48ns2\") pod \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") "
Jul 14 00:21:54.621005 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621008    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tls-certs\") pod \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") "
Jul 14 00:21:54.621308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621028    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tmp-dir\") pod \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") "
Jul 14 00:21:54.621308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621042    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-home\") pod \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") "
Jul 14 00:21:54.621308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621092    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-dshm\") pod \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\" (UID: \"f4ffbc43-30c6-49cc-9db5-9af5106e3dda\") "
Jul 14 00:21:54.621308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621191    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-model-cache" (OuterVolumeSpecName: "model-cache") pod "f4ffbc43-30c6-49cc-9db5-9af5106e3dda" (UID: "f4ffbc43-30c6-49cc-9db5-9af5106e3dda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:54.621308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621285    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.621540 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621335    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f4ffbc43-30c6-49cc-9db5-9af5106e3dda" (UID: "f4ffbc43-30c6-49cc-9db5-9af5106e3dda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:54.621540 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.621351    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-home" (OuterVolumeSpecName: "home") pod "f4ffbc43-30c6-49cc-9db5-9af5106e3dda" (UID: "f4ffbc43-30c6-49cc-9db5-9af5106e3dda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:54.623465 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.623428    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-dshm" (OuterVolumeSpecName: "dshm") pod "f4ffbc43-30c6-49cc-9db5-9af5106e3dda" (UID: "f4ffbc43-30c6-49cc-9db5-9af5106e3dda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:21:54.623465 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.623447    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f4ffbc43-30c6-49cc-9db5-9af5106e3dda" (UID: "f4ffbc43-30c6-49cc-9db5-9af5106e3dda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:21:54.623465 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.623456    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-kube-api-access-48ns2" (OuterVolumeSpecName: "kube-api-access-48ns2") pod "f4ffbc43-30c6-49cc-9db5-9af5106e3dda" (UID: "f4ffbc43-30c6-49cc-9db5-9af5106e3dda"). InnerVolumeSpecName "kube-api-access-48ns2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:21:54.721989 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.721930    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.721989 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.721983    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48ns2\" (UniqueName: \"kubernetes.io/projected/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-kube-api-access-48ns2\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.721989 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.721998    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.722257 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.722012    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.722257 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.722022    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4ffbc43-30c6-49cc-9db5-9af5106e3dda-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:21:54.776886 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:54.776846    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="270d5d29-e38c-4226-9b97-cb65d9b60143" path="/var/lib/kubelet/pods/270d5d29-e38c-4226-9b97-cb65d9b60143/volumes"
Jul 14 00:21:55.032066 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.032030    2550 generic.go:356] "Generic (PLEG): container finished" podID="f4ffbc43-30c6-49cc-9db5-9af5106e3dda" containerID="b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f" exitCode=0
Jul 14 00:21:55.032281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.032080    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"
Jul 14 00:21:55.032281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.032131    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm" event={"ID":"f4ffbc43-30c6-49cc-9db5-9af5106e3dda","Type":"ContainerDied","Data":"b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f"}
Jul 14 00:21:55.032281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.032184    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm" event={"ID":"f4ffbc43-30c6-49cc-9db5-9af5106e3dda","Type":"ContainerDied","Data":"e2cecbad2a198151ebe7803389959df0dc827b75bc96a1b29305594398452873"}
Jul 14 00:21:55.032281 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.032206    2550 scope.go:117] "RemoveContainer" containerID="b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f"
Jul 14 00:21:55.043470 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.043446    2550 scope.go:117] "RemoveContainer" containerID="b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f"
Jul 14 00:21:55.043808 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:21:55.043782    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f\": container with ID starting with b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f not found: ID does not exist" containerID="b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f"
Jul 14 00:21:55.043928 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.043814    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f"} err="failed to get container status \"b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f\": rpc error: code = NotFound desc = could not find container \"b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f\": container with ID starting with b0faa4eee9c9c6c1058e108e6dda267d4e0b6e694f4e79c0aeb6a086d22c609f not found: ID does not exist"
Jul 14 00:21:55.052561 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.052531    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"]
Jul 14 00:21:55.060032 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:55.059997    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/sub-test-model-09117f-kserve-565c6cc6ff-psfrm"]
Jul 14 00:21:56.777563 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:56.777517    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ffbc43-30c6-49cc-9db5-9af5106e3dda" path="/var/lib/kubelet/pods/f4ffbc43-30c6-49cc-9db5-9af5106e3dda/volumes"
Jul 14 00:21:58.648342 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.648298    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"]
Jul 14 00:21:58.648864 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.648843    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ffbc43-30c6-49cc-9db5-9af5106e3dda" containerName="main"
Jul 14 00:21:58.648975 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.648868    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ffbc43-30c6-49cc-9db5-9af5106e3dda" containerName="main"
Jul 14 00:21:58.648975 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.648920    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="270d5d29-e38c-4226-9b97-cb65d9b60143" containerName="main"
Jul 14 00:21:58.648975 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.648929    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="270d5d29-e38c-4226-9b97-cb65d9b60143" containerName="main"
Jul 14 00:21:58.649130 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.649027    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="270d5d29-e38c-4226-9b97-cb65d9b60143" containerName="main"
Jul 14 00:21:58.649130 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.649043    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ffbc43-30c6-49cc-9db5-9af5106e3dda" containerName="main"
Jul 14 00:21:58.653600 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.653575    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.657034 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.657010    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"rate-test-model-7e917f-kserve-self-signed-certs\""
Jul 14 00:21:58.657163 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.657010    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"default-dockercfg-xdrvx\""
Jul 14 00:21:58.657163 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.657018    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"openshift-service-ca.crt\""
Jul 14 00:21:58.657163 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.657067    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e917f\"/\"kube-root-ca.crt\""
Jul 14 00:21:58.661681 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.661564    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"]
Jul 14 00:21:58.757224 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.757180    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-dshm\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.757410 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.757242    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-home\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.757410 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.757267    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-model-cache\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.757410 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.757306    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-tmp-dir\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.757410 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.757323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrsm4\" (UniqueName: \"kubernetes.io/projected/f572aad4-6eb6-414d-a105-1686c297d562-kube-api-access-wrsm4\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.757410 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.757398    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f572aad4-6eb6-414d-a105-1686c297d562-tls-certs\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858259 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858224    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-home\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858440 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858266    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-model-cache\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858440 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858392    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-tmp-dir\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858440 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858421    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrsm4\" (UniqueName: \"kubernetes.io/projected/f572aad4-6eb6-414d-a105-1686c297d562-kube-api-access-wrsm4\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858562 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858458    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f572aad4-6eb6-414d-a105-1686c297d562-tls-certs\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858617 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858574    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-dshm\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858713 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858679    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-home\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858851 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858815    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-model-cache\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.858851 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.858830    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-tmp-dir\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.861017 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.860993    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-dshm\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.861132 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.861114    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f572aad4-6eb6-414d-a105-1686c297d562-tls-certs\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.866655 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.866628    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrsm4\" (UniqueName: \"kubernetes.io/projected/f572aad4-6eb6-414d-a105-1686c297d562-kube-api-access-wrsm4\") pod \"rate-test-model-7e917f-kserve-84c68786ff-6gh72\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") " pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:58.963945 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:58.963805    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:21:59.112862 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:21:59.112826    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"]
Jul 14 00:21:59.116473 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:21:59.116437    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf572aad4_6eb6_414d_a105_1686c297d562.slice/crio-74ef73666e201bdaca5e0a86ba22f78f64ef74ddc962ee9321784951706cb097 WatchSource:0}: Error finding container 74ef73666e201bdaca5e0a86ba22f78f64ef74ddc962ee9321784951706cb097: Status 404 returned error can't find the container with id 74ef73666e201bdaca5e0a86ba22f78f64ef74ddc962ee9321784951706cb097
Jul 14 00:22:00.060489 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:00.060450    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72" event={"ID":"f572aad4-6eb6-414d-a105-1686c297d562","Type":"ContainerStarted","Data":"24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9"}
Jul 14 00:22:00.060489 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:00.060494    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72" event={"ID":"f572aad4-6eb6-414d-a105-1686c297d562","Type":"ContainerStarted","Data":"74ef73666e201bdaca5e0a86ba22f78f64ef74ddc962ee9321784951706cb097"}
Jul 14 00:22:00.078488 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:00.078427    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72" podStartSLOduration=2.078410671 podStartE2EDuration="2.078410671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:22:00.076523727 +0000 UTC m=+2571.874424424" watchObservedRunningTime="2026-07-14 00:22:00.078410671 +0000 UTC m=+2571.876311369"
Jul 14 00:22:02.187228 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:02.187185    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:22:08.963949 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:08.963881    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:22:08.963949 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:08.963959    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:22:08.977796 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:08.977764    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:22:09.118144 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:09.118110    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:22:26.037888 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.037849    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"]
Jul 14 00:22:26.041247 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.041229    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.043728 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.043703    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"kube-root-ca.crt\""
Jul 14 00:22:26.044691 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.044664    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"default-dockercfg-qwpzb\""
Jul 14 00:22:26.044822 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.044700    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"openshift-service-ca.crt\""
Jul 14 00:22:26.044822 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.044667    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-09117f\"/\"rate-test-model-09117f-kserve-self-signed-certs\""
Jul 14 00:22:26.050434 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.050404    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"]
Jul 14 00:22:26.103762 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.103726    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-dshm\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.103973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.103775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tmp-dir\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.103973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.103861    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-home\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.103973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.103893    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-model-cache\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.103973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.103932    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqrpc\" (UniqueName: \"kubernetes.io/projected/a681778d-3649-4cc7-ae13-a5d4a38d07a0-kube-api-access-wqrpc\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.103973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.103968    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tls-certs\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205250 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205215    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-home\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205258    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-model-cache\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205277    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqrpc\" (UniqueName: \"kubernetes.io/projected/a681778d-3649-4cc7-ae13-a5d4a38d07a0-kube-api-access-wqrpc\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205317    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tls-certs\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205354    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-dshm\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205469 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205382    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tmp-dir\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205726 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205666    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-home\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205726 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205714    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-model-cache\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.205831 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.205814    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tmp-dir\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.207877 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.207849    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-dshm\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.208199 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.208179    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tls-certs\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.214331 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.214303    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqrpc\" (UniqueName: \"kubernetes.io/projected/a681778d-3649-4cc7-ae13-a5d4a38d07a0-kube-api-access-wqrpc\") pod \"rate-test-model-09117f-kserve-dfd59c887-sr6t2\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") " pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.352156 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.352038    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:26.494411 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:26.494382    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"]
Jul 14 00:22:26.496345 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:22:26.496313    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda681778d_3649_4cc7_ae13_a5d4a38d07a0.slice/crio-1a47477dced30d1a80c6fcb09c20be3b12e03365ac148ff913640867962484ea WatchSource:0}: Error finding container 1a47477dced30d1a80c6fcb09c20be3b12e03365ac148ff913640867962484ea: Status 404 returned error can't find the container with id 1a47477dced30d1a80c6fcb09c20be3b12e03365ac148ff913640867962484ea
Jul 14 00:22:27.201526 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:27.201481    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2" event={"ID":"a681778d-3649-4cc7-ae13-a5d4a38d07a0","Type":"ContainerStarted","Data":"e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7"}
Jul 14 00:22:27.201526 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:27.201532    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2" event={"ID":"a681778d-3649-4cc7-ae13-a5d4a38d07a0","Type":"ContainerStarted","Data":"1a47477dced30d1a80c6fcb09c20be3b12e03365ac148ff913640867962484ea"}
Jul 14 00:22:27.217994 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:27.217941    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2" podStartSLOduration=1.21792499 podStartE2EDuration="1.21792499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:22:27.216502905 +0000 UTC m=+2599.014403603" watchObservedRunningTime="2026-07-14 00:22:27.21792499 +0000 UTC m=+2599.015825679"
Jul 14 00:22:31.380886 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:31.380795    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:22:36.352923 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:36.352851    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:36.352923 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:36.352937    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:36.365728 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:36.365694    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:37.262786 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:37.262751    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:22:52.124547 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:22:52.124511    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:23:07.070856 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:07.070818    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:23:13.722326 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:13.722287    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"]
Jul 14 00:23:14.725389 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:14.725345    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"]
Jul 14 00:23:21.481749 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:21.481709    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:23:28.743657 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:28.743600    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72" podUID="f572aad4-6eb6-414d-a105-1686c297d562" containerName="main" containerID="cri-o://24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9" gracePeriod=45
Jul 14 00:23:28.998959 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:28.998879    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:23:29.073757 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.073716    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrsm4\" (UniqueName: \"kubernetes.io/projected/f572aad4-6eb6-414d-a105-1686c297d562-kube-api-access-wrsm4\") pod \"f572aad4-6eb6-414d-a105-1686c297d562\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") "
Jul 14 00:23:29.073757 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.073762    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-home\") pod \"f572aad4-6eb6-414d-a105-1686c297d562\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") "
Jul 14 00:23:29.074059 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.073778    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-tmp-dir\") pod \"f572aad4-6eb6-414d-a105-1686c297d562\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") "
Jul 14 00:23:29.074059 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.073808    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-dshm\") pod \"f572aad4-6eb6-414d-a105-1686c297d562\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") "
Jul 14 00:23:29.074059 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.073829    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-model-cache\") pod \"f572aad4-6eb6-414d-a105-1686c297d562\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") "
Jul 14 00:23:29.074059 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.073960    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f572aad4-6eb6-414d-a105-1686c297d562-tls-certs\") pod \"f572aad4-6eb6-414d-a105-1686c297d562\" (UID: \"f572aad4-6eb6-414d-a105-1686c297d562\") "
Jul 14 00:23:29.074308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.074090    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-home" (OuterVolumeSpecName: "home") pod "f572aad4-6eb6-414d-a105-1686c297d562" (UID: "f572aad4-6eb6-414d-a105-1686c297d562"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:29.074308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.074102    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f572aad4-6eb6-414d-a105-1686c297d562" (UID: "f572aad4-6eb6-414d-a105-1686c297d562"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:29.074308 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.074217    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-model-cache" (OuterVolumeSpecName: "model-cache") pod "f572aad4-6eb6-414d-a105-1686c297d562" (UID: "f572aad4-6eb6-414d-a105-1686c297d562"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:29.074420 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.074319    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:29.074420 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.074336    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:29.074420 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.074351    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:29.076245 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.076205    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-dshm" (OuterVolumeSpecName: "dshm") pod "f572aad4-6eb6-414d-a105-1686c297d562" (UID: "f572aad4-6eb6-414d-a105-1686c297d562"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:29.076354 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.076248    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f572aad4-6eb6-414d-a105-1686c297d562-kube-api-access-wrsm4" (OuterVolumeSpecName: "kube-api-access-wrsm4") pod "f572aad4-6eb6-414d-a105-1686c297d562" (UID: "f572aad4-6eb6-414d-a105-1686c297d562"). InnerVolumeSpecName "kube-api-access-wrsm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:23:29.076354 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.076259    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f572aad4-6eb6-414d-a105-1686c297d562-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f572aad4-6eb6-414d-a105-1686c297d562" (UID: "f572aad4-6eb6-414d-a105-1686c297d562"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:23:29.175187 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.175130    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f572aad4-6eb6-414d-a105-1686c297d562-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:29.175187 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.175175    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrsm4\" (UniqueName: \"kubernetes.io/projected/f572aad4-6eb6-414d-a105-1686c297d562-kube-api-access-wrsm4\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:29.175187 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.175186    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f572aad4-6eb6-414d-a105-1686c297d562-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:29.503225 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.503184    2550 generic.go:356] "Generic (PLEG): container finished" podID="f572aad4-6eb6-414d-a105-1686c297d562" containerID="24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9" exitCode=0
Jul 14 00:23:29.503387 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.503235    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"
Jul 14 00:23:29.503387 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.503263    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72" event={"ID":"f572aad4-6eb6-414d-a105-1686c297d562","Type":"ContainerDied","Data":"24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9"}
Jul 14 00:23:29.503387 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.503313    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72" event={"ID":"f572aad4-6eb6-414d-a105-1686c297d562","Type":"ContainerDied","Data":"74ef73666e201bdaca5e0a86ba22f78f64ef74ddc962ee9321784951706cb097"}
Jul 14 00:23:29.503387 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.503338    2550 scope.go:117] "RemoveContainer" containerID="24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9"
Jul 14 00:23:29.513309 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.513289    2550 scope.go:117] "RemoveContainer" containerID="24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9"
Jul 14 00:23:29.513599 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:23:29.513577    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9\": container with ID starting with 24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9 not found: ID does not exist" containerID="24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9"
Jul 14 00:23:29.513697 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.513608    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9"} err="failed to get container status \"24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9\": rpc error: code = NotFound desc = could not find container \"24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9\": container with ID starting with 24e2dff58b918ceee4e88a0ae13b3cb1ebf50ee7961579dfd2427ad75f4ac2f9 not found: ID does not exist"
Jul 14 00:23:29.523800 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.523771    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"]
Jul 14 00:23:29.527926 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.527884    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e917f/rate-test-model-7e917f-kserve-84c68786ff-6gh72"]
Jul 14 00:23:29.746606 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.746538    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2" podUID="a681778d-3649-4cc7-ae13-a5d4a38d07a0" containerName="main" containerID="cri-o://e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7" gracePeriod=45
Jul 14 00:23:29.996887 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:29.996856    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:23:30.084765 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.084734    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-home\") pod \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") "
Jul 14 00:23:30.084765 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.084772    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-dshm\") pod \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") "
Jul 14 00:23:30.085062 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.084808    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tmp-dir\") pod \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") "
Jul 14 00:23:30.085062 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.084866    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqrpc\" (UniqueName: \"kubernetes.io/projected/a681778d-3649-4cc7-ae13-a5d4a38d07a0-kube-api-access-wqrpc\") pod \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") "
Jul 14 00:23:30.085062 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.084933    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-model-cache\") pod \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") "
Jul 14 00:23:30.085062 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.084971    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tls-certs\") pod \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\" (UID: \"a681778d-3649-4cc7-ae13-a5d4a38d07a0\") "
Jul 14 00:23:30.085274 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.085122    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-home" (OuterVolumeSpecName: "home") pod "a681778d-3649-4cc7-ae13-a5d4a38d07a0" (UID: "a681778d-3649-4cc7-ae13-a5d4a38d07a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:30.085274 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.085152    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a681778d-3649-4cc7-ae13-a5d4a38d07a0" (UID: "a681778d-3649-4cc7-ae13-a5d4a38d07a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:30.085395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.085281    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "a681778d-3649-4cc7-ae13-a5d4a38d07a0" (UID: "a681778d-3649-4cc7-ae13-a5d4a38d07a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:30.085395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.085323    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-home\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:30.085395 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.085342    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tmp-dir\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:30.087311 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.087287    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a681778d-3649-4cc7-ae13-a5d4a38d07a0" (UID: "a681778d-3649-4cc7-ae13-a5d4a38d07a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 00:23:30.087311 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.087292    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-dshm" (OuterVolumeSpecName: "dshm") pod "a681778d-3649-4cc7-ae13-a5d4a38d07a0" (UID: "a681778d-3649-4cc7-ae13-a5d4a38d07a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:23:30.087459 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.087291    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a681778d-3649-4cc7-ae13-a5d4a38d07a0-kube-api-access-wqrpc" (OuterVolumeSpecName: "kube-api-access-wqrpc") pod "a681778d-3649-4cc7-ae13-a5d4a38d07a0" (UID: "a681778d-3649-4cc7-ae13-a5d4a38d07a0"). InnerVolumeSpecName "kube-api-access-wqrpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:23:30.186021 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.185969    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-model-cache\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:30.186021 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.186013    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a681778d-3649-4cc7-ae13-a5d4a38d07a0-tls-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:30.186021 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.186022    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a681778d-3649-4cc7-ae13-a5d4a38d07a0-dshm\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:30.186021 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.186032    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqrpc\" (UniqueName: \"kubernetes.io/projected/a681778d-3649-4cc7-ae13-a5d4a38d07a0-kube-api-access-wqrpc\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:30.510747 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.510652    2550 generic.go:356] "Generic (PLEG): container finished" podID="a681778d-3649-4cc7-ae13-a5d4a38d07a0" containerID="e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7" exitCode=0
Jul 14 00:23:30.510747 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.510702    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"
Jul 14 00:23:30.510962 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.510741    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2" event={"ID":"a681778d-3649-4cc7-ae13-a5d4a38d07a0","Type":"ContainerDied","Data":"e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7"}
Jul 14 00:23:30.510962 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.510788    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2" event={"ID":"a681778d-3649-4cc7-ae13-a5d4a38d07a0","Type":"ContainerDied","Data":"1a47477dced30d1a80c6fcb09c20be3b12e03365ac148ff913640867962484ea"}
Jul 14 00:23:30.510962 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.510810    2550 scope.go:117] "RemoveContainer" containerID="e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7"
Jul 14 00:23:30.522159 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.522139    2550 scope.go:117] "RemoveContainer" containerID="e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7"
Jul 14 00:23:30.522509 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:23:30.522485    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7\": container with ID starting with e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7 not found: ID does not exist" containerID="e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7"
Jul 14 00:23:30.522644 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.522522    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7"} err="failed to get container status \"e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7\": rpc error: code = NotFound desc = could not find container \"e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7\": container with ID starting with e7b58b251814264a56a3a7cf23a028545778972fc353baea0f6d2179e929ecd7 not found: ID does not exist"
Jul 14 00:23:30.531961 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.531930    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"]
Jul 14 00:23:30.536088 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.536064    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-09117f/rate-test-model-09117f-kserve-dfd59c887-sr6t2"]
Jul 14 00:23:30.776790 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.776710    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a681778d-3649-4cc7-ae13-a5d4a38d07a0" path="/var/lib/kubelet/pods/a681778d-3649-4cc7-ae13-a5d4a38d07a0/volumes"
Jul 14 00:23:30.777373 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:30.777167    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f572aad4-6eb6-414d-a105-1686c297d562" path="/var/lib/kubelet/pods/f572aad4-6eb6-414d-a105-1686c297d562/volumes"
Jul 14 00:23:31.046846 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.046806    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7504-140612647085088"]
Jul 14 00:23:31.047180 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.047167    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a681778d-3649-4cc7-ae13-a5d4a38d07a0" containerName="main"
Jul 14 00:23:31.047233 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.047181    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a681778d-3649-4cc7-ae13-a5d4a38d07a0" containerName="main"
Jul 14 00:23:31.047233 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.047193    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f572aad4-6eb6-414d-a105-1686c297d562" containerName="main"
Jul 14 00:23:31.047233 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.047199    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f572aad4-6eb6-414d-a105-1686c297d562" containerName="main"
Jul 14 00:23:31.047341 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.047266    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f572aad4-6eb6-414d-a105-1686c297d562" containerName="main"
Jul 14 00:23:31.047341 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.047273    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a681778d-3649-4cc7-ae13-a5d4a38d07a0" containerName="main"
Jul 14 00:23:31.051499 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.051474    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:31.053891 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.053870    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-9295c\""
Jul 14 00:23:31.057197 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.057175    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7504-140612647085088"]
Jul 14 00:23:31.194582 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.194538    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svfmx\" (UniqueName: \"kubernetes.io/projected/fc06a6ff-40b7-4729-b4ca-22ce22855bef-kube-api-access-svfmx\") pod \"test-curl-7504-140612647085088\" (UID: \"fc06a6ff-40b7-4729-b4ca-22ce22855bef\") " pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:31.296075 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.296029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svfmx\" (UniqueName: \"kubernetes.io/projected/fc06a6ff-40b7-4729-b4ca-22ce22855bef-kube-api-access-svfmx\") pod \"test-curl-7504-140612647085088\" (UID: \"fc06a6ff-40b7-4729-b4ca-22ce22855bef\") " pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:31.305641 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.305568    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svfmx\" (UniqueName: \"kubernetes.io/projected/fc06a6ff-40b7-4729-b4ca-22ce22855bef-kube-api-access-svfmx\") pod \"test-curl-7504-140612647085088\" (UID: \"fc06a6ff-40b7-4729-b4ca-22ce22855bef\") " pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:31.361296 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.361256    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:31.502262 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.502223    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7504-140612647085088"]
Jul 14 00:23:31.505584 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:23:31.505535    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc06a6ff_40b7_4729_b4ca_22ce22855bef.slice/crio-66c70c33685b0c1dc5e475e95fc56d440c0ef61b3e91956f630e5e8f19d86e6a WatchSource:0}: Error finding container 66c70c33685b0c1dc5e475e95fc56d440c0ef61b3e91956f630e5e8f19d86e6a: Status 404 returned error can't find the container with id 66c70c33685b0c1dc5e475e95fc56d440c0ef61b3e91956f630e5e8f19d86e6a
Jul 14 00:23:31.507344 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.507307    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 00:23:31.520820 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:31.520760    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7504-140612647085088" event={"ID":"fc06a6ff-40b7-4729-b4ca-22ce22855bef","Type":"ContainerStarted","Data":"66c70c33685b0c1dc5e475e95fc56d440c0ef61b3e91956f630e5e8f19d86e6a"}
Jul 14 00:23:32.532814 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:32.532772    2550 generic.go:356] "Generic (PLEG): container finished" podID="fc06a6ff-40b7-4729-b4ca-22ce22855bef" containerID="5b7ba7265b96aeef789ea86dcd2945e922aed7fb1b202fcdd92cbf5094c57ba2" exitCode=0
Jul 14 00:23:32.533250 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:32.532838    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7504-140612647085088" event={"ID":"fc06a6ff-40b7-4729-b4ca-22ce22855bef","Type":"ContainerDied","Data":"5b7ba7265b96aeef789ea86dcd2945e922aed7fb1b202fcdd92cbf5094c57ba2"}
Jul 14 00:23:33.684003 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.683971    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:33.720490 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.720439    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7504-140612647085088_fc06a6ff-40b7-4729-b4ca-22ce22855bef/test-curl-7504-140612647085088/0.log"
Jul 14 00:23:33.742237 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.742198    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7504-140612647085088"]
Jul 14 00:23:33.745309 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.745280    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7504-140612647085088"]
Jul 14 00:23:33.820202 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.820096    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svfmx\" (UniqueName: \"kubernetes.io/projected/fc06a6ff-40b7-4729-b4ca-22ce22855bef-kube-api-access-svfmx\") pod \"fc06a6ff-40b7-4729-b4ca-22ce22855bef\" (UID: \"fc06a6ff-40b7-4729-b4ca-22ce22855bef\") "
Jul 14 00:23:33.822350 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.822315    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc06a6ff-40b7-4729-b4ca-22ce22855bef-kube-api-access-svfmx" (OuterVolumeSpecName: "kube-api-access-svfmx") pod "fc06a6ff-40b7-4729-b4ca-22ce22855bef" (UID: "fc06a6ff-40b7-4729-b4ca-22ce22855bef"). InnerVolumeSpecName "kube-api-access-svfmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:23:33.921219 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:33.921174    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svfmx\" (UniqueName: \"kubernetes.io/projected/fc06a6ff-40b7-4729-b4ca-22ce22855bef-kube-api-access-svfmx\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:23:34.544374 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:34.544339    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66c70c33685b0c1dc5e475e95fc56d440c0ef61b3e91956f630e5e8f19d86e6a"
Jul 14 00:23:34.544561 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:34.544378    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:34.554058 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:23:34.554023    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7504-140612647085088\" is forbidden: User \"system:node:ip-10-0-137-167.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-137-167.ec2.internal' and this object" podUID="fc06a6ff-40b7-4729-b4ca-22ce22855bef" pod="openshift-ingress/test-curl-7504-140612647085088"
Jul 14 00:23:34.776533 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:34.776491    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc06a6ff-40b7-4729-b4ca-22ce22855bef" path="/var/lib/kubelet/pods/fc06a6ff-40b7-4729-b4ca-22ce22855bef/volumes"
Jul 14 00:23:47.880012 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:23:47.879971    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:24:08.878664 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:08.878572    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:24:08.886957 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:08.886930    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:24:08.888164 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:08.888143    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:24:08.895771 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:08.895750    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zrpr4_cf7a5c33-e26b-48a5-bb02-a50ec358a582/ovn-acl-logging/0.log"
Jul 14 00:24:13.480267 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:13.480223    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pvwzq"]
Jul 14 00:24:25.909404 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:25.909368    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"]
Jul 14 00:24:25.910010 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:25.909654    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy" containerID="cri-o://d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56" gracePeriod=30
Jul 14 00:24:26.637502 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:26.637466    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9bd75549b-tnplk_0431311a-49af-49d7-8d87-3c7d2df6208d/authorino/0.log"
Jul 14 00:24:31.168739 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.168704    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:24:31.243025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.242883    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-istio-token\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.242978    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-socket\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243021    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-certs\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243065    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5ba35b9-5280-42ba-974b-80823becdceb-istiod-ca-cert\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243087    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lqj4\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-kube-api-access-4lqj4\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243134    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5ba35b9-5280-42ba-974b-80823becdceb-istio-podinfo\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243166    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-credential-socket\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243212    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-data\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243430 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243247    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-envoy\") pod \"b5ba35b9-5280-42ba-974b-80823becdceb\" (UID: \"b5ba35b9-5280-42ba-974b-80823becdceb\") "
Jul 14 00:24:31.243730 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243587    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:24:31.243730 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243646    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:24:31.243730 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243674    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5ba35b9-5280-42ba-974b-80823becdceb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 00:24:31.243869 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.243748    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:24:31.244599 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.244561    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:24:31.245535 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.245492    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:24:31.245687 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.245660    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5ba35b9-5280-42ba-974b-80823becdceb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 14 00:24:31.245785 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.245712    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 00:24:31.245785 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.245760    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-kube-api-access-4lqj4" (OuterVolumeSpecName: "kube-api-access-4lqj4") pod "b5ba35b9-5280-42ba-974b-80823becdceb" (UID: "b5ba35b9-5280-42ba-974b-80823becdceb"). InnerVolumeSpecName "kube-api-access-4lqj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 00:24:31.344069 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344026    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-data\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344069 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344060    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-istio-envoy\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344069 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344072    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-istio-token\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344084    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344098    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-workload-certs\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344109    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5ba35b9-5280-42ba-974b-80823becdceb-istiod-ca-cert\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344120    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lqj4\" (UniqueName: \"kubernetes.io/projected/b5ba35b9-5280-42ba-974b-80823becdceb-kube-api-access-4lqj4\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344133    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5ba35b9-5280-42ba-974b-80823becdceb-istio-podinfo\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.344338 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.344144    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5ba35b9-5280-42ba-974b-80823becdceb-credential-socket\") on node \"ip-10-0-137-167.ec2.internal\" DevicePath \"\""
Jul 14 00:24:31.832041 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.832000    2550 generic.go:356] "Generic (PLEG): container finished" podID="b5ba35b9-5280-42ba-974b-80823becdceb" containerID="d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56" exitCode=0
Jul 14 00:24:31.832240 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.832065    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"
Jul 14 00:24:31.832240 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.832064    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" event={"ID":"b5ba35b9-5280-42ba-974b-80823becdceb","Type":"ContainerDied","Data":"d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56"}
Jul 14 00:24:31.832240 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.832122    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9" event={"ID":"b5ba35b9-5280-42ba-974b-80823becdceb","Type":"ContainerDied","Data":"01eb7fa3fb278d0d88814410c0582d4ed8eeb8f8e0f80ca7941c1b2b152bbc5b"}
Jul 14 00:24:31.832240 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.832147    2550 scope.go:117] "RemoveContainer" containerID="d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56"
Jul 14 00:24:31.841582 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.841562    2550 scope.go:117] "RemoveContainer" containerID="a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"
Jul 14 00:24:31.850693 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.850664    2550 scope.go:117] "RemoveContainer" containerID="d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56"
Jul 14 00:24:31.851380 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:24:31.851345    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56\": container with ID starting with d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56 not found: ID does not exist" containerID="d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56"
Jul 14 00:24:31.851515 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.851389    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56"} err="failed to get container status \"d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56\": rpc error: code = NotFound desc = could not find container \"d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56\": container with ID starting with d8828eb033eae86b83aad4b2169cbc7f41803d64064bff79983544c18e9d1f56 not found: ID does not exist"
Jul 14 00:24:31.851515 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.851416    2550 scope.go:117] "RemoveContainer" containerID="a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"
Jul 14 00:24:31.851786 ip-10-0-137-167 kubenswrapper[2550]: E0714 00:24:31.851760    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4\": container with ID starting with a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4 not found: ID does not exist" containerID="a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"
Jul 14 00:24:31.851857 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.851792    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4"} err="failed to get container status \"a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4\": rpc error: code = NotFound desc = could not find container \"a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4\": container with ID starting with a1f2918127f424d7d2cbab9c3d702c7db214b33b8e1fe7d01cd5aa371d9ad1b4 not found: ID does not exist"
Jul 14 00:24:31.853516 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.853493    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"]
Jul 14 00:24:31.856760 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.856734    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-09117f-openshift-default-9bbcf89ff-z4gm9"]
Jul 14 00:24:31.907429 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:31.907397    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-96thl_dc0ec160-222f-48b2-8217-c316cc26ed8e/manager/0.log"
Jul 14 00:24:32.673095 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:32.673066    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74bf4db56c-rmhth_37d78dee-ed79-4803-8926-4af54d0d5719/copy-manifests/0.log"
Jul 14 00:24:32.679402 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:32.679378    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74bf4db56c-rmhth_37d78dee-ed79-4803-8926-4af54d0d5719/manager/0.log"
Jul 14 00:24:32.777681 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:32.777644    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" path="/var/lib/kubelet/pods/b5ba35b9-5280-42ba-974b-80823becdceb/volumes"
Jul 14 00:24:33.469202 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:33.469168    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-0bcce182-5gfq2_7bb4508d-ec32-4735-94d8-8e8811a11dfe/revoke-keys/0.log"
Jul 14 00:24:33.587619 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:33.587588    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-30780878-4m89b_ca727a49-5e41-4a46-882f-7a5b2d33497c/revoke-keys/0.log"
Jul 14 00:24:33.708383 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:33.708351    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-211a7b-f6bdp_88bdd9f1-d2d6-4782-a60f-7cb12108aac1/revoke-keys/0.log"
Jul 14 00:24:34.458637 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:34.458608    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-45127b63-prmtx_82fd1b1a-1ee1-4340-9157-78be3df03587/revoke-keys/0.log"
Jul 14 00:24:36.235702 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:36.235670    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9bd75549b-tnplk_0431311a-49af-49d7-8d87-3c7d2df6208d/authorino/0.log"
Jul 14 00:24:36.941999 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:36.941966    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pvwzq_db071360-e86c-4023-8da2-786494ea079a/limitador/0.log"
Jul 14 00:24:37.414574 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:37.414544    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l9l96_39270a84-2b24-471a-a015-b0071a524724/istio-proxy/0.log"
Jul 14 00:24:37.651111 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:37.651082    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-764dccbb5b-4f2kx_8e37dd35-277a-4806-be76-13c59c1493df/kube-auth-proxy/0.log"
Jul 14 00:24:38.273121 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:38.273091    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676457fb64-74c8b_4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d/router/0.log"
Jul 14 00:24:43.046925 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.046878    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kbgg9/must-gather-chl24"]
Jul 14 00:24:43.047403 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047256    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc06a6ff-40b7-4729-b4ca-22ce22855bef" containerName="test-curl-7504-140612647085088"
Jul 14 00:24:43.047403 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047270    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc06a6ff-40b7-4729-b4ca-22ce22855bef" containerName="test-curl-7504-140612647085088"
Jul 14 00:24:43.047403 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047280    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy"
Jul 14 00:24:43.047403 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047285    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy"
Jul 14 00:24:43.047403 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047355    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy"
Jul 14 00:24:43.047403 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047364    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc06a6ff-40b7-4729-b4ca-22ce22855bef" containerName="test-curl-7504-140612647085088"
Jul 14 00:24:43.047617 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047418    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy"
Jul 14 00:24:43.047617 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047424    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy"
Jul 14 00:24:43.047617 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.047480    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5ba35b9-5280-42ba-974b-80823becdceb" containerName="istio-proxy"
Jul 14 00:24:43.051837 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.051810    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.054738 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.054685    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kbgg9\"/\"openshift-service-ca.crt\""
Jul 14 00:24:43.055617 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.055574    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kbgg9\"/\"kube-root-ca.crt\""
Jul 14 00:24:43.055693 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.055618    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kbgg9\"/\"default-dockercfg-5fbwc\""
Jul 14 00:24:43.059598 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.059535    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kbgg9/must-gather-chl24"]
Jul 14 00:24:43.149686 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.149639    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt8wr\" (UniqueName: \"kubernetes.io/projected/44ab05d5-b4c1-4ae4-970c-5379febad258-kube-api-access-vt8wr\") pod \"must-gather-chl24\" (UID: \"44ab05d5-b4c1-4ae4-970c-5379febad258\") " pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.149871 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.149736    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/44ab05d5-b4c1-4ae4-970c-5379febad258-must-gather-output\") pod \"must-gather-chl24\" (UID: \"44ab05d5-b4c1-4ae4-970c-5379febad258\") " pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.251025 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.250977    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/44ab05d5-b4c1-4ae4-970c-5379febad258-must-gather-output\") pod \"must-gather-chl24\" (UID: \"44ab05d5-b4c1-4ae4-970c-5379febad258\") " pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.251237 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.251045    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt8wr\" (UniqueName: \"kubernetes.io/projected/44ab05d5-b4c1-4ae4-970c-5379febad258-kube-api-access-vt8wr\") pod \"must-gather-chl24\" (UID: \"44ab05d5-b4c1-4ae4-970c-5379febad258\") " pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.251381 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.251357    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/44ab05d5-b4c1-4ae4-970c-5379febad258-must-gather-output\") pod \"must-gather-chl24\" (UID: \"44ab05d5-b4c1-4ae4-970c-5379febad258\") " pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.259400 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.259370    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt8wr\" (UniqueName: \"kubernetes.io/projected/44ab05d5-b4c1-4ae4-970c-5379febad258-kube-api-access-vt8wr\") pod \"must-gather-chl24\" (UID: \"44ab05d5-b4c1-4ae4-970c-5379febad258\") " pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.366925 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.366802    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kbgg9/must-gather-chl24"
Jul 14 00:24:43.508941 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.508891    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kbgg9/must-gather-chl24"]
Jul 14 00:24:43.511205 ip-10-0-137-167 kubenswrapper[2550]: W0714 00:24:43.511173    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44ab05d5_b4c1_4ae4_970c_5379febad258.slice/crio-3472950bc31835e3667d8a69a23cc78e4e4a7e0a915623e3615c25182a9cad11 WatchSource:0}: Error finding container 3472950bc31835e3667d8a69a23cc78e4e4a7e0a915623e3615c25182a9cad11: Status 404 returned error can't find the container with id 3472950bc31835e3667d8a69a23cc78e4e4a7e0a915623e3615c25182a9cad11
Jul 14 00:24:43.892347 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:43.892307    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kbgg9/must-gather-chl24" event={"ID":"44ab05d5-b4c1-4ae4-970c-5379febad258","Type":"ContainerStarted","Data":"3472950bc31835e3667d8a69a23cc78e4e4a7e0a915623e3615c25182a9cad11"}
Jul 14 00:24:44.901554 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:44.901511    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kbgg9/must-gather-chl24" event={"ID":"44ab05d5-b4c1-4ae4-970c-5379febad258","Type":"ContainerStarted","Data":"997e224aa04d9b5a72f3c4f17573e613256877b5cbee17fd5e5d8fa58de5a7d1"}
Jul 14 00:24:44.902204 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:44.902182    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kbgg9/must-gather-chl24" event={"ID":"44ab05d5-b4c1-4ae4-970c-5379febad258","Type":"ContainerStarted","Data":"ca9686a16a5a4d351e4198a6513400ac9f729fcc616e37035ec2468f157099b5"}
Jul 14 00:24:44.916817 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:44.916756    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kbgg9/must-gather-chl24" podStartSLOduration=1.032582794 podStartE2EDuration="1.916734386s" totalImagesPullingTime="884.151592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:24:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:24:44.916470282 +0000 UTC m=+2736.714370981" watchObservedRunningTime="2026-07-14 00:24:44.916734386 +0000 UTC m=+2736.714635085"
Jul 14 00:24:45.940166 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:45.940126    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4z7b5_7f86be0b-cc85-4452-b67d-a4d942ebfa31/global-pull-secret-syncer/0.log"
Jul 14 00:24:46.103519 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:46.103488    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m4zpw_d603b112-cd8d-4348-849c-76050a6d3ee9/konnectivity-agent/0.log"
Jul 14 00:24:46.216177 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:46.216060    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-167.ec2.internal_15816fd0954517b7da05c28e845e91a4/haproxy/0.log"
Jul 14 00:24:51.104511 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:51.104468    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9bd75549b-tnplk_0431311a-49af-49d7-8d87-3c7d2df6208d/authorino/0.log"
Jul 14 00:24:51.428362 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:51.428327    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pvwzq_db071360-e86c-4023-8da2-786494ea079a/limitador/0.log"
Jul 14 00:24:53.305068 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.305027    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-slmmw_ae0a023b-56d5-4aa5-8b90-5acc5e7829b5/cluster-monitoring-operator/0.log"
Jul 14 00:24:53.685105 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.685072    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lkx8k_a748fa36-27cd-4aa8-aeb6-b59617b0ea7f/node-exporter/0.log"
Jul 14 00:24:53.710793 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.710758    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lkx8k_a748fa36-27cd-4aa8-aeb6-b59617b0ea7f/kube-rbac-proxy/0.log"
Jul 14 00:24:53.740225 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.740187    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lkx8k_a748fa36-27cd-4aa8-aeb6-b59617b0ea7f/init-textfile/0.log"
Jul 14 00:24:53.854311 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.854279    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/prometheus/0.log"
Jul 14 00:24:53.885711 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.885673    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/config-reloader/0.log"
Jul 14 00:24:53.907707 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.907672    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/thanos-sidecar/0.log"
Jul 14 00:24:53.929652 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.929626    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/kube-rbac-proxy-web/0.log"
Jul 14 00:24:53.953379 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.953348    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/kube-rbac-proxy/0.log"
Jul 14 00:24:53.975144 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.975115    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/kube-rbac-proxy-thanos/0.log"
Jul 14 00:24:53.998290 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:53.998247    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e4f392f-07f6-4b16-acde-051c04c3de95/init-config-reloader/0.log"
Jul 14 00:24:55.718657 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:55.718603    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-wdhn7_c0e8ab74-34a5-40ec-911e-e186ea2a05df/networking-console-plugin/0.log"
Jul 14 00:24:56.388522 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:56.388473    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/1.log"
Jul 14 00:24:56.396234 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:56.396193    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-8pn7m_2f9d2f69-2b34-48e8-8a7d-b6ab20563f66/console-operator/2.log"
Jul 14 00:24:56.963346 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:56.963311    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-mblkq_6b55064c-10d2-49eb-ae25-6838d1080089/download-server/0.log"
Jul 14 00:24:57.249647 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.249545    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"]
Jul 14 00:24:57.258493 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.258446    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.263164 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.263125    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"]
Jul 14 00:24:57.400138 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.400084    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-sys\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.400358 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.400175    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-lib-modules\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.400358 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.400211    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm96j\" (UniqueName: \"kubernetes.io/projected/66288b9a-4b78-4883-9971-0d24ede22f4b-kube-api-access-qm96j\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.400358 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.400317    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-proc\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.400511 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.400403    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-podres\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.501496 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.501352    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-proc\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.501496 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.501479    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-podres\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.501759 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.501525    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-sys\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.501759 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.501589    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-lib-modules\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.501759 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.501620    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm96j\" (UniqueName: \"kubernetes.io/projected/66288b9a-4b78-4883-9971-0d24ede22f4b-kube-api-access-qm96j\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.502155 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.502125    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-proc\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.502296 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.502275    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-podres\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.502359 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.502341    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-sys\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.502483 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.502461    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66288b9a-4b78-4883-9971-0d24ede22f4b-lib-modules\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.511678 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.511631    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm96j\" (UniqueName: \"kubernetes.io/projected/66288b9a-4b78-4883-9971-0d24ede22f4b-kube-api-access-qm96j\") pod \"perf-node-gather-daemonset-r2vmc\" (UID: \"66288b9a-4b78-4883-9971-0d24ede22f4b\") " pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.576973 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.576925    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:57.682322 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.682288    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-6btql_cee06c03-b815-4b87-b0b1-dd2c86353e9b/volume-data-source-validator/0.log"
Jul 14 00:24:57.770305 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:57.769237    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"]
Jul 14 00:24:58.014651 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.014607    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc" event={"ID":"66288b9a-4b78-4883-9971-0d24ede22f4b","Type":"ContainerStarted","Data":"65ddf4412c7dda871ecc868190854ca26fee16af56dde6f1f9712180465ed686"}
Jul 14 00:24:58.015381 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.014677    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc" event={"ID":"66288b9a-4b78-4883-9971-0d24ede22f4b","Type":"ContainerStarted","Data":"2da5198cdbb2c14b2f8a24b0d47c35870fab65833936695f04189f768334277b"}
Jul 14 00:24:58.015381 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.014723    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:24:58.031932 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.031844    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc" podStartSLOduration=1.031819177 podStartE2EDuration="1.031819177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:24:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:24:58.030304407 +0000 UTC m=+2749.828205130" watchObservedRunningTime="2026-07-14 00:24:58.031819177 +0000 UTC m=+2749.829719876"
Jul 14 00:24:58.772509 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.772479    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5796w_6136130f-72a5-4fa2-8624-a2013d9c2f9f/dns/0.log"
Jul 14 00:24:58.797628 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.797587    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5796w_6136130f-72a5-4fa2-8624-a2013d9c2f9f/kube-rbac-proxy/0.log"
Jul 14 00:24:58.911349 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:58.911315    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6gp4k_83cb2404-9a86-4938-a281-f5fd33be5f71/dns-node-resolver/0.log"
Jul 14 00:24:59.609807 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:24:59.609771    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rgzpl_d79c4dfe-ae6c-470f-b0a6-700348765f66/node-ca/0.log"
Jul 14 00:25:00.539325 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:00.539287    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l9l96_39270a84-2b24-471a-a015-b0071a524724/istio-proxy/0.log"
Jul 14 00:25:00.764161 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:00.764121    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-764dccbb5b-4f2kx_8e37dd35-277a-4806-be76-13c59c1493df/kube-auth-proxy/0.log"
Jul 14 00:25:00.976152 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:00.976059    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676457fb64-74c8b_4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d/router/0.log"
Jul 14 00:25:01.766686 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:01.766654    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z8cxq_99cafe45-8468-4bf3-854b-476ff78603d4/serve-healthcheck-canary/0.log"
Jul 14 00:25:02.237423 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:02.237368    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-c8c6g_ed419e48-23c4-41c4-a1f1-37c3b07f7eba/insights-operator/1.log"
Jul 14 00:25:02.244480 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:02.244448    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-c8c6g_ed419e48-23c4-41c4-a1f1-37c3b07f7eba/insights-operator/0.log"
Jul 14 00:25:02.327989 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:02.327951    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7xwz_07a4af18-785a-4876-81d0-ef941ff8a4fe/kube-rbac-proxy/0.log"
Jul 14 00:25:02.347933 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:02.347883    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7xwz_07a4af18-785a-4876-81d0-ef941ff8a4fe/exporter/0.log"
Jul 14 00:25:02.368614 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:02.368588    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7xwz_07a4af18-785a-4876-81d0-ef941ff8a4fe/extractor/0.log"
Jul 14 00:25:04.032374 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:04.031480    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kbgg9/perf-node-gather-daemonset-r2vmc"
Jul 14 00:25:04.712277 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:04.712240    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-96thl_dc0ec160-222f-48b2-8217-c316cc26ed8e/manager/0.log"
Jul 14 00:25:05.170549 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:05.170504    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74bf4db56c-rmhth_37d78dee-ed79-4803-8926-4af54d0d5719/manager/0.log"
Jul 14 00:25:05.190472 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:05.190439    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74bf4db56c-rmhth_37d78dee-ed79-4803-8926-4af54d0d5719/copy-manifests/0.log"
Jul 14 00:25:06.532809 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:06.532717    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5dc78dff7b-7tzdb_3db368c6-e1dd-4bfa-85d7-6932d54de99d/manager/0.log"
Jul 14 00:25:07.976183 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:07.976097    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l9l96_39270a84-2b24-471a-a015-b0071a524724/istio-proxy/0.log"
Jul 14 00:25:08.197032 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:08.197003    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-764dccbb5b-4f2kx_8e37dd35-277a-4806-be76-13c59c1493df/kube-auth-proxy/0.log"
Jul 14 00:25:08.374858 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:08.374828    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676457fb64-74c8b_4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d/router/0.log"
Jul 14 00:25:09.777864 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:09.777834    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678l9l96_39270a84-2b24-471a-a015-b0071a524724/istio-proxy/0.log"
Jul 14 00:25:09.992222 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:09.992194    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-764dccbb5b-4f2kx_8e37dd35-277a-4806-be76-13c59c1493df/kube-auth-proxy/0.log"
Jul 14 00:25:10.165332 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:10.165204    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-676457fb64-74c8b_4dbc64b3-d49c-4a9f-8242-eae3a0ac1e0d/router/0.log"
Jul 14 00:25:12.269449 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:12.269405    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-7xjhz_8691f032-2217-4c94-b168-144e74a61f5b/kube-storage-version-migrator-operator/1.log"
Jul 14 00:25:12.270315 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:12.270297    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-7xjhz_8691f032-2217-4c94-b168-144e74a61f5b/kube-storage-version-migrator-operator/0.log"
Jul 14 00:25:13.357754 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.357724    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/kube-multus-additional-cni-plugins/0.log"
Jul 14 00:25:13.379748 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.379712    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/egress-router-binary-copy/0.log"
Jul 14 00:25:13.401399 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.401357    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/cni-plugins/0.log"
Jul 14 00:25:13.424015 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.423983    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/bond-cni-plugin/0.log"
Jul 14 00:25:13.443540 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.443507    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/routeoverride-cni/0.log"
Jul 14 00:25:13.463217 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.463190    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/whereabouts-cni-bincopy/0.log"
Jul 14 00:25:13.482912 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.482881    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4kw9l_5deee0cd-e42a-4244-ae33-a0207f26dfb1/whereabouts-cni/0.log"
Jul 14 00:25:13.902244 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.902210    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kkdxd_02caab57-ed2b-44ed-b831-fe9aaaaeaba6/kube-multus/0.log"
Jul 14 00:25:13.932072 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.932040    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4mr6t_7426e531-b898-4126-ae24-cc3925a6c20a/network-metrics-daemon/0.log"
Jul 14 00:25:13.950531 ip-10-0-137-167 kubenswrapper[2550]: I0714 00:25:13.950498    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4mr6t_7426e531-b898-4126-ae24-cc3925a6c20a/kube-rbac-proxy/0.log"